Schizophrenia Research 143 (2013) 143-149


Contents lists available at SciVerse ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres


On the HORYZON: Moderated online social therapy for long-term recovery in first episode psychosis

M. Alvarez-Jimenez '1,l),:1 S. Bendall a,b, R. Lederman c, G. Wadleyc, G. Chinnery d, S. Vargas d, M. Larkin a,b E. Killackey a,b, P.D. McGorry a,b, J.F. Gleeson e

a Centre for Youth Mental Health, The University of Melbourne, Australia

b Orygen Youth Health Research Centre, Australia

c The Department of Computing and Information Systems, The University of Melbourne, Australia

d Orygen Youth Health, Australia

e Australian Catholic University, School of Psychology, Melbourne, Australia

ARTICLE INFO


ABSTRACT


Article history:

Received 14 June 2012

Received in revised form 28 August 2012

Accepted 7 October 2012

Available online 9 November 2012


Background: Early intervention services have demonstrated improved outcomes in first episode psychosis (FEP); however, recent evidence shows that treatment benefits may not be sustainable over time. These findings have resulted in repeated recommendations for the implementation of longer term treatment programs. An Internet-based intervention specifically designed for young people with psychosis may provide a cost-effective alternative to prevent loss of treatment benefits from early intervention.

Keywords:

First episode psychosis

Early intervention

Online

Long-term recovery

Schizophrenia

Social networking


Methods: Our multi-disciplinary team has developed a highly novel online intervention (HORYZONS) in regular consultation with stakeholders within a specialist early psychosis program. HORYZONS integrates: i) peer-to-peer social networking, ii) individually tailored interactive psychosocial interventions, and iii) expert interdisciplinary and peer-moderation in a coherent platform designed to improve long-term outcomes in FEP. The acceptability, safety and initial clinical benefits of HORYZONS were examined through a 1-month pilot study with 20 participants with FEP.

Results: There were no dropouts during the pilot study. Seventy per cent of participants utilised the system for at least 3 weeks, 95% used the social networking features, and 60% completed at least 3 therapy modules. System usage was high during the study. There were no incidents and the majority of participants reported feeling safe, empowered and more socially connected using HORYZONS. Analysis revealed a significant reduction in depressive symptoms at follow-up.

Conclusions: Our results indicate that HORYZONS is feasible, engaging and safe and may augment social connectedness and empowerment in FEP. These findings have significant implications for the enhancement of specialist FEP services. The potential of HORYZONS to improve long-term recovery is worthy of further investigation.

© 2012 Elsevier B.V. All rights reserved.

1. Introduction

Over the preceding two decades, early intervention for psychosis has emerged as a major international psychiatric reform (McGorry and Yung, 2003). The transformation of psychiatric care has led to the proliferation of specialist first episode psychosis (FEP) services in Australia, Europe, North America and Asia (McGorry et al., 2008). Early psychosis services seek to provide timely entry into treatment and support young people in achieving both symptomatic remission and full functional recovery (McGorry et al., 1996). They adopt a youth-friendly and recovery-focussed model in order to empower young people to become more involved in their own treatment decisions and prevent the development of self-stigma, comorbid symptomatology and functional and social disability (McGorry et al., 2008). Controlled and quasi-experimental research has demonstrated that specialist FEP services improve psychotic symptoms (Petersen et al., 2005), reduce relapse rates (Craig et al., 2004; Petersen et al., 2005; Alvarez-Jimenez et al., 2011) and comorbid substance misuse (Petersen et al., 2005), and foster functional recovery (Melle et al., 2004), engagement with services (Craig et al., 2004) and patient satisfaction (Petersen et al., 2005), at least during the first 2 years of treatment.

However, the maintenance of treatment effects poses a major challenge to achieving the aims of early intervention (Singh, 2010; Gleeson et al., 2011). Patients with FEP are responsive to acute-phase treatments, but relapse rates are high (Alvarez-Jimenez et al., 2012a). Our group has developed effective cognitive-based interventions for relapse prevention in FEP (Gleeson et al., 2009), but treatment benefits were lost over time suggesting the interventions needed to be maintained over the longer term (Gleeson et al., 2011). More generally, specialist FEP services typically have treatment resources for only 18 months to 2 years, and recent reports suggest that the benefits of early intervention seen at the end of 2 years may not persist at 5 years after patients have been receiving generic follow-up (Bertelsen et al., 2008; Gafoor et al., 2010). These findings have triggered widespread calls for studies assessing the effects of longer term treatment programs (Bertelsen et al., 2008; Friis, 2010; Singh, 2010).

0920-9964/$ - see front matter © 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.schres.2012.10.009


The positive effects of specialist FEP services are likely to persist when patients continue to receive specialised treatment (Linszen et al., 2001; Norman et al., 2011). This view is supported by the evidence that the first 5 years after psychosis onset constitutes a critical period determining longer term outcomes (Crumlish et al., 2009; Norman et al., 2011). Indeed, the period of risk for relapse extends well beyond the timeframe of specialist FEP services (Alvarez-Jimenez et al., 2012a). Furthermore, the termination of the specialised intervention and the transfer of care bring about feelings of loss for the patients (Bertelsen et al., 2008) and significantly derail engagement with treatment services (Singh, 2010), a pivotal element of early intervention programmes.

In this context, there is a clear need to fully realise the aims of early intervention by extending its effectiveness through innovative approaches (Kreyenbuhl et al., 2009; Singh, 2010). A stepped down level of care which bridges the gap between specialised intervention and standard treatment could provide a cost-effective alternative to prevent loss of treatment benefits and reduce the risk for disengagement from mental health services over the longer term (Norman and Malla, 2001). Internet-based interventions have the potential to realise this goal through providing cost-effective, non-stigmatising, constantly available support to young people suffering from psychosis.

This article aims to: i) provide a rationale for Internet-based interventions for psychosis and their potential to improve long-term recovery in FEP; ii) describe the development of a world-first online psychosocial intervention (HORYZONS) designed to maintain the benefits of early intervention beyond discharge from specialist FEP services; and iii) examine the feasibility, acceptability, safety and potential clinical benefits of HORYZONS in FEP patients through a pilot study.

2. Internet-based technologies and early intervention for psychosis

2.1. Are online interventions for first episode psychosis feasible?

The Internet and mobile technologies are developing at a phenomenal rate and hold great promise for influencing and even transforming treatment delivery in psychosis (Ben-Zeev, 2012). These technologies can be particularly useful for young people with FEP as individuals under 25 are by far the greatest users of Internet resources (Lloyd and Bill, 2004). Indeed, 95% of young Australians use the Internet daily (Ewing et al., 2008) and 96% of people aged 16-24 in the UK have used the Internet, with 86% using it every day or almost every day (Sharkey et al., 2011). Likewise, our data show that FEP patients show positive attitudes toward Internet-based interventions and, in particular, toward online social networking (Lederman et al., 2011).

2.2. What type of Internet-based intervention?

To date, despite their clinical and social potential, only a handful of studies have investigated the effects of evidence-based interventions for psychosis delivered through Internet-based technologies. These include online forums, computer-based psychoeducation, and mobilebased interventions.

2.2.1. Online peer-support groups

The development of peer support or user-led services is based on solid theory as well as promising research findings (Davidson et al., 2006). Peer support rests on the assumption that people who have overcome adversity can provide valuable support, guidance and hope to others facing similar difficulties (Davidson et al., 2006). Moreover, consistent with the helper-therapy principle (Riessman, 1965), being able to be of assistance to others can improve self-esteem and reduce experiences of self-stigma (Corrigan, 2006). Research on user-led services has found robust associations between peer support, empowerment, social networks and recovery in people suffering from psychosis (Corrigan, 2006). Social support has also shown to be a protective factor against psychotic relapse in FEP (Norman et al., 2005). While research on the effects of online peer-support groups for people with psychosis is sparse, preliminary evidence suggests that professionally moderated forums focussed on positive constructs such as self-efficacy, problem solving and social recovery are likely to produce the best outcomes (Alvarez-Jimenez et al., 2012b).

2.2.2. Online psychoeducation

Psychoeducation interventions for psychosis have been shown to improve medication compliance, reduce relapse rates, promote social functioning, and increase satisfaction with mental health services (Xia et al., 2011). Preliminary research further suggests that computerbased psychoeducation is acceptable, as effective as face-to-face or bibliotherapy, and preferred by some (Jones et al., 2001; Walker, 2006). Although more research is needed to ascertain their effective elements, online interventions which provide clear and engaging information and cater for level of insight are likely to produce better outcomes (Williams and Whitfield, 2001; Gonzalez-Blanch et al., 2007; Alvarez-Jimenez et al., 2009).

2.2.3. Mobile-based interventions

The characteristics of novel mobile devices including their accessibility, portability, affordability, online-connectivity, and ease of use provide an opportunity to widely deliver evidence-based interventions and enable real-time support in psychosis treatment (Alvarez-Jimenez et al., 2012b). Indeed, recent research shows that therapeutic interventions employing mobile devices are both acceptable to and efficiently used by people suffering from psychosis (Depp et al., 2010). Moreover, preliminary evidence shows that mobile phones and SMS-based interventions can be successfully used to monitor early warning signs (EWS) of relapse and prevent hospital admissions (Spaniel et al., 2008) as well as significantly improve medication adherence, socialisation and auditory hallucinations (Granholm et al., 2012).

2.2.4. Internet-based interventions and FEP

To date, no study has investigated the feasibility or effects of Internet-based interventions for FEP patients. Consequently, we conducted a focus group with young people with psychosis to examine their views on technology-based tools (Alvarez-Jimenez et al., 2012b). Qualitative analysis showed that FEP patients were enthusiastic about the use of Internet-based strategies as part of their treatment program. They clearly preferred a system characterised by social connectivity (i.e. users should be able to contact peers, share personal experiences, and contact clinicians if required). The system should resemble regular social networking programs (i.e. asynchronous, ongoing communication), but provide a private and enclosed social network. In addition, expert moderators should guide, but not censor, the interaction to ensure a safe and supportive environment. Finally, patients indicated that the system should provide useful and updated information relevant to their needs, and optional therapeutic interventions (i.e. cognitive-behavioural strategies). Conversely, concerns were raised about the potential risks of Internet-based strategies when experiencing active psychotic symptoms and about content which made them focus negatively on their condition (Alvarez-Jimenez et al., 2012b).

2.3. How to develop an Internet-based intervention for FEP patients?

A number of recommendations have been put forward to increase the likelihood of acceptance, safety and effectiveness of Internetbased interventions in psychosis (Valimaki et al., 2008). These include careful analysis of users' needs and preferences prior to development, active involvement of stakeholders throughout all phases of development (Valimaki et al., 2008), and regular consultations with potential users with regards to all aspects of the protocols (i.e. emergency responding, therapy/psychoeducation content and wording and specific features) (Depp et al., 2010). Careful attention should be paid to clinical safety and emergency procedures (Sharkey et al., 2011). Additionally, the relevance of multimodal induction procedures and pilot testing has been highlighted (Depp et al., 2010). Ultimately, Internet-based interventions for FEP that are specifically designed to supplement existing mental health services and augment traditional relationships through online interaction are likely to be most beneficial (Alvarez-Jimenez et al., 2012b).

3. Development of the MOST model and the HORYZONS system

In line with the aforementioned recommendations, we have developed a new conceptual model of on-line behavioural interventions entitled “Moderated On-line Social Therapy" (MOST). The MOST model integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) involvement of expert mental health and peer moderators to ensure the safety of the intervention. The elements of the MOST model have been applied to a world-first on-line system entitled HORYZONS, designed with the purpose of maintaining the clinical benefits of early intervention.

HORYZONS has been developed by a multidisciplinary team of clinical psychologists, computer programmers, health informatics experts, web and graphic designers, and professional writers. In accordance with international recommendations, the online system has been developed iteratively over a 30-month period following participatory design principles (Schuler and Namioka, 1993) through a process of regular feedback and testing with focus groups of stakeholders (i.e. FEP patients and clinicians). HORYZONS includes fully functional versions for computer and Internet-enabled mobile devices (i.e. smart-phones and tablets).

3.1. Characteristics of HORYZONS

The HORYZONS system adopts the MOST conceptual model of online interventions. Thus, HORYZONS comprises a platform for the delivery of a range of evidence-based and interactive psychosocial interventions which are enhanced by a moderated on-line social networking environment. Specifically, HORYZONS integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) expert moderation.

3.1.1. Interactive psychosocial interventions

On entry into the system the patient answers a set of standardised questions that guide the delivery of tailored interactive psychoeducation modules. Modules are informed by evidence-based psychosocial interventions developed by our group (Killackey et al., 2008; Gleeson et al., 2009) and target key risk factors for psychotic relapse and salient domains in the early recovery process including: (a) psychoeducation, (b) early warning signs of relapse (EWS), (c) depression, (d) social anxiety and (e) stress management (Table 1). Some interactive modules are mandatory (i.e. psycho-education or How minds work') while others are optional (e.g. Early Warning Signs' (EWS) of Relapse) and their delivery is guided by user preferences. In addition, the content of modules is tailored to individual clinical characteristics. For example, specifically designed psychoeducation" or EWS" modules have been developed for participants with high insight vs. low insight to foster their engagement with the system (Alvarez-Jimenez et al., 2009). Finally, the therapeutic objectives of the modules are bolstered through discussion with peers in the cafe(online environment) under the guidance of coaches' (expert moderators).

3.1.2. Peer-to-peer on-line social networking (the cafe)

The cafeincludes a web feed (or newsfeed) where clients and moderators can post comments, information, upload pictures and videos, and likedifferent content. The newsfeed incorporates categories that organise discussion threads into relevant themes (e.g. what's on your mind, I'm loving right now, strengths newsgroup news) (eFig. 1). Moreover the system includes a wall’ function displaying the activity of individual users, a homepage’ which shows all the activity and notifications relevant to the user, a network(similar to a friendsfunction) and a job zonewhere users are provided with information regarding training and vocational recovery and access online to a vocational rehabilitation expert (eFig. 2). HORYZONS further incorporates a number of moderated social networking features that are progressively unlocked as patients advance through the therapy modules. For example, participants are invited to join moderated online problem solving groups. Nominated issues are discussed through structured phases following an evidence-based problem solving framework (McFarlane et al., 1995; McFarlane, 2002). Offered solutions and users' experiences are stored in a data-base serving as a wikifor participants. Moreover, participants are encouraged to share personal coping resources for stressful events (i.e. what works for me) which are accumulated and categorised for future reference.

Table 1

HORYZONS therapy modules.a

Module

Description

%b

Snapshot

This module included an interactive evaluation of patient's goals and a number questions assessing level of insight and recovery style, anxiety and depression symptoms and social anxiety and stigma. At the end of the module HORYZONS automatically generated a formulation letter or snapshot'(Ryle and Kerr, 2003) which summarised patients feedback and linked their responses to specific HORYZONS modules and social networking features.

90%

How minds work

This module provided interactive information on psychosis and the recovery process. Information was provided in a non-stigmatizing and positive manner with particular emphasis on empowerment and social recovery. Different themes were illustrated through 3 fictional characters.

70%

Strengths

In this module patients were introduced with the concept of personal strengths. An interactive online card sort game helped users identify their signature' strengths. Assessment of strengths was informed by positive psychology framework (Seligman et al., 2006; Rashid and Ostermann, 2009). Participants were encouraged to share their experiences using strengths in the online social networking.

60%

Early warning

signs (EWS)

This module included an interactive online card sort game to identify potential EWS of relapse and categorize them into early, middle and late signs (eFig. 4). Subsequently, a relapse prevention plan (RPT) was developed interactively in which coping strategies and therapeutic techniques were linked to groups of EWS (Birchwood et al., 1989).

45%

The comfort zone

This module included interactive exercises to help patients identify their current level of activity (comfort zone') and promote gradual exposure and increased activity (expanding the zone').

20%

Ninja thinking skills

This module included a number of cognitive-based strategies to help patients identify unhelpful thinking patterns (i.e. ruminative thoughts, paranoia) and promote action-focussed, real, concrete thoughts. The concept of flow was also introduced to illustrate helpful thinking patterns (Csikszentmihalyi, 1990).

20%

To the HORYZONS

The final module consisted of a general overview of the key aspects of the completed modules. This module emphasized personal achievement, provided recommendations to stay well' and encouraged participants to use the social networking features and practice personal strengths on a regular basis.

15%

a All modules included talking pointswhere participants were encouraged to share their own ideas and experiences and wrap-upswhich summarised the take home messages.

b Percentage of participants of completed the module.


Importantly, HORYZONS adopts a strength-based approach (Seligman et al., 2006) through which users are guided and prompted to identify, discuss and exercise key personal strengths within the online environment and in real-life to enhance self-esteem, foster social functioning (Hall and Tarrier, 2003) and reduce depression (Seligman et al., 2006) (eFig. 3). This approach is based on the novel positive psychology model which poses that psychosocial interventions should aim to build strengths, meaning and purpose as well as relieve symptoms (Lee Duckworth etal., 2005; Seligman et al., 2006). Preliminary research suggests that this model is particularly appropriate for online interventions. For example, FEP patients may not engage effectively in online content which make them focus negatively on their condition (Alvarez-Jimenez et al., 2012b). Further, excessive expressions of fear and anxiety online increase stress and reduce quality of life (Lieberman and Goldstein, 2006), while Internet interventions focussed on self-efficacy have produced improved clinical outcomes in patients with psychosis (Rotondi et al., 2010).

3.1.3. Expert moderation

HORYZONS uniquely provides a hubin which clinical psychologists and vocational workers collaboratively moderate the system with the purpose of improving both clinical and vocational functioning. Their role is to provide guidance, monitor participant's clinical status and ensure the safety of the social networking environment. HORYZONS moderation follows a theory-driven model known as supportive accountabilitywhich poses that human support enhances engagement through accountability to a moderator who is perceived as trustworthy, benevolent and having expertise. Accountability involves clear, process-focussed and user driven expectations that take into account patient motivation (i.e. level of support is inversely proportional to the patient's intrinsic motivation) (Mohr et al., 2011). HORYZONS is moderated on a daily basis (i.e. 2 h/day during week-days, and 1 h/day during weekends).

3.1.4. HORYZONS safety protocol

The HORYZONS safety protocol follows best practice in Internet research involving vulnerable people (Sharkey et al., 2011) and is composed of 3 levels of security including system security, online safety, and clinical safety. System security conforms to industry best practice as defined by the Open Web Application Security Project (OWASP). Online safety is managed in accordance with the guidelines put forward by the Australian Communications and Media Authority (ACMA).

Clinical safety is ensured through manual and automated procedures. First, information related to clinical risk is screened daily by moderators. Second, HORYZONS incorporates an automatic alert system which detects information consistent with increased risk of relapse or suicide (via regular monitoring of EWS of relapse and self-harm-related terms using previously validated approaches (Goh and Huang, 2009; Huang et al., 2007). Any detected increased risk activates the HORYZONS crisis protocol that includes an initial risk assessment and, where necessary, liaison with suitable emergency services. Details on the safety protocol and ethical issues have been provided elsewhere (Gleeson et al., in press).

4. The HORYZONS pilot study

The purpose of the pilot study was to conduct an initial evaluation of HORYZONS regarding its feasibility, acceptability, safety and potential clinical utility for FEP patients. We hypothesised that HORYZONS would be: 1) feasible and favourably received; 2) regularly used; 3) safe; and 4) viewed as a useful tool for long-term recovery and social connectedness.

4.1. Participants

The sample included 20 patients recruited from the Early Psychosis Prevention and Intervention Centre (EPPIC, Melbourne). Inclusion criteria for the study were: (a) a first episode of a DSM-IV (APA, 1994) psychotic disorder or mood disorder with psychotic features; (b) aged 15-25 years inclusive; (c) <6 months treatment with an antipsychotic medication prior to registration with the early psychosis service; and (d) remission of positive symptoms of psychosis, defined as scores of 3 (mild) or below on the subscale items hallucinations, unusual thought disorder, conceptual disorganisation, and suspiciousness on the Brief Psychiatric Rating Scale (BPRS) (Lukoff et al., 1986) over that last 4 weeks. Additional inclusion criteria to ensure low level of risk within HORYZONS included: (e) low aggressiveness, defined by a score of 4 or below on the hostility subscale of the expanded version of the BPRS for the month prior to study entry; and (f) low suicidal risk defined as a score of 4 or below on the BPRS suicidality subscale for the month preceding study entry. Exclusion criteria included: (a) intellectual disability and (b) inability to converse in, or read English. Additional exclusion criteria to ensure safety within the online system included (c) a DSM-IV (APA, 1994) diagnosis of either antisocial personality disorder (ASPD) or (d) borderline personality disorder (BPD).

The mean age was 20.3 years (SD = 2.7) with 50% male. Selfreported ethnicity was 45% Anglo Australian, 25% Asian, 15% European, 5% African and 10% Bi-racial. Thirty five per cent of the samples were unemployed with 100% never married. Ninety per cent of the samples were living with their family and 10% living with friends. The study was approved by the Melbourne Health Human Research Ethics Committee. Participants provided informed consent while continuing treatment within the EPPIC programme.

4.2. Design and procedures

This study used an uncontrolled single-group design to evaluate feasibility, acceptability, safety and initial benefits of a pilot psychosocial intervention (Mueser and Drake, 2005). Study participants were recruited over a 3-week period. Eligible participants were invited to participate by the study research assistant (RA) after the online application was briefly showcased. Once informed consent was obtained, the RA provided participants with logon details, helped them set up their account and oriented them to the HORYZONS system, including details of the terms of use. Online moderators welcomed new users and encouraged existing users to interact with them within 24 h of enrolment.

HORYZONS was monitored daily (i.e. 2 h/day during week-days, and 1 h/day during weekends) by clinical psychologists and a vocational worker. Moderation integrity was ensured through a detailed moderation manual and weekly group supervision sessions with senior clinical researchers (MA-J and JF-G) from the research team. Participants were assessed at baseline and at 4 weeks follow-up on outcomes described below.

4.3. Measures

Baseline DSM-IV diagnosis was assessed via the Structured Clinical Interview for DSM-IV (SCID patient version) (First et al., 1996). Feasibility and acceptance of HORYZONS were tracked measuring participants' usage of the online system (i.e. frequency, duration and patterns of use). In addition, a questionnaire and semi-structured interview were developed based on the User Experience (UX) approach (Bargas-Avila and Hornbsk, 2011) which assessed the following themes: 1) helpfulness; 2) easy-of-use; 3) attractiveness; 4) safety; 5) social interaction; and 6) further suggestions. Each domain was assessed through a number of statements which the respondent rated on a Likert scale from strongly agree to strongly disagree as well as open-ended questions. Good acceptability of the system was defined a priori as at least 10 of the 20 participants logging on weekly during the 1-month pilot.

Safety was assessed through the following indicators: 1) adverse events (i.e. psychotic relapse and clinical deterioration related to usage of the system), 2) inappropriate use, and 3) participants' reports on inappropriate content.

Symptom rating measures included the 24-item version of the Brief Psychiatric Rating Scale (BPRS) (Lukoff et al., 1986), the clinician-rated Calgary Depression Scale for Schizophrenia (CDSS) (Addington et al., 1993), and the Beck Anxiety Inventory (BAI) (Beck et al., 1988).

4.4. Analysis

Frequency, duration and patterns of use were tracked in real time. Up-to-date charts were visible on demand to researchers and moderators via the website. Quantitative and qualitative feedbacks from the questionnaire were analysed and reviewed for themes related to usability, perceived benefits, safety, and challenges of online interventions. Paired samples t-tests were conducted and within-group effect sizes reported for statistically significant changes between baseline and post-test in clinical measures.

4.5. Results

4.5.1. Feasibility and acceptance

Data on the usage of HORYZONS showed that 60% of participants utilised the system over 4 weeks and 70% for at least 3 out of 4 weeks, with a total of 275 logins during the trial (Table 2). Participants provided positive ratings for the ease of use, perceived utility, and enjoyment of HORYZONS. The majority (75%) reported that they had a positive and constructive experience using HORYZONS, 90% would recommend it to others, and no participants reported negative or challenging experiences.

The social networking features were used by 95% of the sample, with a total of 371 postings and 4170 social page views or actions (i.e. posts, like) during the study. There was a median of 192 (range = 0-702) social pages views/actions per participant (Table 2). Accordingly, 70% perceived social networking as useful, and 90% considered moderation to be supportive. Interestingly, 85% considered that it would be beneficial to include moderators who were previous users of HORYZONS and 90% reported they would like to become online peer-moderators.

With respect to therapy module usage, 95% completed at least 1 full therapy module, 60% completed 3 at least modules, 45% completed 4 modules, and 15% completed all 7 modules (Table 1). There were a total of 1390 therapy-based page views, with a median of 65 (range = 7-169) per participant (Table 2).

4.5.2. Safety

No incidents (i.e. adverse events or inappropriate usage) were reported during the study. Similarly, analysis showed no worsening of positive psychotic symptoms at 1-month follow-up (Table 3). One participant experienced a psychotic relapse; however, both the participant and the treating team considered the clinical worsening unrelated to the use of HORYZONS. This is further supported by the low usage of HORYZONS by the relapsing participant, with only 2 logins during the trial. Finally, 100% participants considered HORYZONS to be both safe and confidential, and 90% reported that moderation contributed to the safety of the system.

4.5.3. Potential clinical utility

Analysis of symptom rating measures revealed a moderate to large improvement in participants' depressive symptoms at 1-month follow-up (Table 2). Moreover, 60% reported that HORYZONS significantly increased their social connectedness, 55% felt empowered in their own recovery process, and 70% considered the system to be a useful long-term treatment option beyond discharge.

5. Discussion

To the best of our knowledge, this is the first study to develop and test an online intervention specifically designed for FEP patients, and is possibly a world-first intervention using the MOST model. HORYZONS has been developed by a multidisciplinary team of experts in constant consultation with stakeholders. It uniquely integrates peer-to-peer online social networking, evidence-based interventions and professional and peer support in a coherent, innovative platform designed to provide ongoing support and prevent disengagement from mental health services. The results of this pilot study demonstrated HORYZONS to be highly acceptable and appealing to young FEP patients. More than two thirds of the sample regularly used the online system and all participants reported feeling safe and supported in doing so. This was further reflected in the highly successful rate of recruitment to the study and the volume of traffic on the site.

Data on system usage and therapy completion strongly supported the feasibility of HORYZONS. There were no drop-outs during the pilot trial and completion of therapy modules was relatively high, with 60% of participants completing at least 3 full therapy modules within one month. This compares well with the standard weekly dosage of psychotherapy provided in individual therapy studies for FEP (Jackson et al., 2005, 2007; Edwards et al., 2006; Gleeson et al., 2009; Penn et al., 2010), particularly considering that HORYZONS

Table 2

Logins and individual usage of the main components of HORYZONS (n = 20) over the 1-month pilot study.a

Site component

M

S.D.

Mdn

Range

%

Logins

13.50

11.95

7.50

1-39

70b

Social networkingc

208.50

191.99

134.50

0-702

95d

Therapy modulesc

69.50

47.60

65.50

7-169

75d

a Covers period from February 1,2012, to 16th of March 16,2012; participants were recruited into the system from February 1, 2012, to February 17, 2012.

b Percentage of participants with more than 6 logins.

c Number of page reads (page hits).

d Percentage of participants with more than 30 page hits or posts.

Table 3

Means (M), standard deviation (S.D.), and within-group effect sizes (Cohen's d) for clinical measures (n = 20).

Baseline

1 month follow-up

p

D

M

S.D.

M

S.D.

BPRS

33.75

7.70

33.45

8.24

0.80

CDRS

4.85

5.95

3.55

5.00

0.02

0.60

BAI

15.95

10.87

12.80

11.81

0.15

BPRS = BriefPsychiatric Rating Scale; CDRS = Calgary Depression Rating Scale; BAI = Beck Anxiety Inventory.

participants undertake therapy modules independently and without the active prompts of therapeutic sessions.

Our attrition and module completion rates compare very favourably to unsupported, open-access, short-term Internet-based programs (Eysenbach, 2005; Christensen et al., 2006), and even online interventions incorporating support and tracking by interviewers or counselors (Christensen et al., 2004; Clarke et al., 2005). Likewise, usage of HORYZONS (i.e. frequency of use in relation to time) compared well to a previous web-based intervention for people with schizophrenia which included online educational materials and a therapy forum (Rotondi et al., 2010). While the short duration of the trial and the characteristics of the clinical sample may partially account for these positive findings, these results suggest that the novel integration of online moderation, peer-to-peer social networking and therapy modules (i.e. the MOST model) provides a promising and engaging platform for delivering online interventions in FEP.

Analysis of clinical variables showed a moderate to large reduction in depressive symptoms at 1-month follow-up (Table 2). Subsequent analysis of individual-level data revealed that 7 of 10 participants with reduced depression scores were regular users of HORYZONS. Whereas the uncontrolled design of the study does not allow causal attributions, the clinical significance of the latter finding should be noted. Depression is pervasive in the early course of psychosis (Upthegrove et al., 2010) and individual interventions for FEP have shown only modest effects in reducing depressive symptoms (Jackson et al., 2005; Edwards et al., 2006; Gleeson et al., 2009, 2011). Taken together, this suggests that possibly the increased social connectedness (as reported by 60% of participants) or the empowering and strengths-based model adopted by HORYZONS (55% of participants reported feeling empowered in their own recovery using the online system) may have contributed to the reduction in depressive symptoms at follow-up. The potential of HORYZONS to improve depressive symptoms in FEP is worthy of further investigation.

The current study has several limitations. First, as noted above, the uncontrolled design precluded any causal inferences about the efficacy of HORYZONS (i.e. reduction in depression may be a function of either time or background interventions), however this was never the intended purpose of the study. Second, the short-term duration of the study precluded examination of the long-term effects and retention of the intervention. Thirdly, the current findings regarding acceptability, safety and potential clinical benefits can only be generalised to remitted FEP patients with low risk of aggression and suicidality. Fourth, the semi-structured interview was administered by an assessor non-blind to the study hypothesis, which may have over-estimated positive findings, and participant responses were subject to demand effects, social desirability and recall bias. That said, considerable efforts were made to minimise these biases. Namely, i) prior to interviews it was emphasised that honest responses were most beneficial for future users and versions of HORYZONS; ii) interviews were conducted showing HORYZONS through an Internet-enabled tablet to minimise recall bias; iii) all interviews were recorded, transcribed and rated by an independent rater; and iv) system usage, qualitative data and responses to the questionnaire were matched for consistency. Notwithstanding these caveats, our pilot study provided proof of conceptfor HORYZONS in terms of its acceptability and potential clinical benefits and warrants a full-scale trial in which these limitations can be addressed.

5.1. Conclusions and future research

The results of this study showed HORYZONS to be a promising online intervention for FEP patients as it yielded high acceptance and usage, low attrition, high satisfaction, and perceived increased social connectedness and empowerment. Participants reported feeling safe and perceived HORYZONS as a useful treatment strategy beyond discharge from a specialised FEP service.

In the near future we will evaluate whether HORYZONS is an effective strategy for the maintenance of specialised treatment effects in FEP. In the meantime the results of our pilot study highlight the potential of HORYZONS to: i) significantly improve the clinical and cost-effectiveness of specialised FEP services; ii) improve access and ongoing engagement of FEP patients in evidence-based preventative interventions; and iii) and transform current models of treatment maintenance in FEP.

Role of funding Source

This study was supported by generous funding from the Telstra Foundation, the Helen MacPherson Smith Trust, the University of Melbourne, the Telematics Trust, The Institute for a Broadband-Enabled Society (IBES), and the Colonial Foundation. The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.

Contributors

MA-J and JF-G supervised the study and wrote the first draft of the manuscript. S-B, R-L, G-W, G-C, S-V and M-L significantly contributed to the development and evaluation of HORYZONS. E-K and PD-M contributed to the design of the study and critically revised the manuscript. All authors contributed to and have approved the final manuscript.

Conflict of interest

The authors report no additional financial or other affiliation relevant to the subject of this article.

Supplementary data to this article can be found online at http://dx.doi.org/10. 1016/j.schres.2012.10.009.

Acknowledgements

The authors wish to thank Christopher Miles, Kathryn Junor and Sylvia Collinety for their contribution to the development of HORYZONS, and the study participants for their valuable contributions to this research.

References

ACMA, Australian Communications Media Authority, y. www.acma.gov.au.

Addington, D., Addington, J., Matickatyndale, E., 1993. Assessing depression in schizophrenia the Calgary Depression Scale. Br. J. Psychiatry 163, 39-44.

Alvarez-Jimenez, M., Gleeson, J.F., Cotton, S., Wade, D., Gee, D., Pearce, T., Crisp, K., Spiliotacopoulos, D., Newman, B., McGorry, P.D., 2009. Predictors of adherence to cognitive-behavioural therapy in first-episode psychosis. Can. J. Psychiatry 54 (10), 710-718.

Alvarez-Jimenez, M., Parker, A.G., Hetrick, S.E., McGorry, P.D., Gleeson, J.F., 2011. Preventing the second episode: a systematic review and meta-analysis of psychosocial and pharmacological trials in first-episode psychosis. Schizophr. Bull. 37 (3), 619-630.

Alvarez-Jimenez, M., Priede, A., Hetrick, S.E., Bendall, S., Killackey, E., Parker, A.G., McGorry, P.D., Gleeson, J.F., 2012a. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr. Res. (as).

Alvarez-Jimenez, M., Gleeson, J.F., Bendall, S., Lederman, R., Wadley, G., Killackey, E., McGorry, P.D., 2012b. Internet-based interventions for psychosis: a sneak-peek into the future. Psychiatr. Clin. North Am. 35 (3), 735-747.

APA, 1994. Diagnostic and Statistical Manual of Mental Disorders, 4 ed. American Psychiatric Association, Washington, DC.

Bargas-Avila, J.A., Hornbsk, K.P., 2011. Old Wine in New Bottles or Novel Challenges? A Critical Analysis of Empirical Studies of User Experience. ACM CHI Conference, Vancouver.

Beck, A.T., Epstein, N., Brown, G., Steer, R.A., 1988. An inventory for measuring clinical anxiety: psychometric properties. J. Consult. Clin. Psychol. 56 (6), 893-897.

Ben-Zeev, D., 2012. Mobile technologies in the study, assessment, and treatment of schizophrenia. Schizophr. Bull. 38 (3), 384-385.

Bertelsen, M., Jeppesen, P., Petersen, L., Thorup, A., Ohlenschlaeger, J., le Quach, P., Christensen, T.O., Krarup, G., Jorgensen, P., Nordentoft, M., 2008. Five-year follow-up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Arch. Gen. Psychiatry 65 (7), 762-771.

Birchwood, M., Smith, J., Macmillan, F., Hogg, B., Prasad, R., Harvey, C., Bering, S., 1989. Predicting relapse in schizophrenia: the development and implementation of an early signs monitoring system using patients and families as observers, a preliminary investigation. Psychol. Med. 19 (3), 649-656.

Christensen, H., Griffiths, K.M., Korten, A.E., Brittliffe, K., Groves, C., 2004. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J. Med. Internet Res. 6 (4), e46.

Christensen, H., Griffiths, K.M., Mackinnon, A.J., Brittliffe, K., 2006. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol. Med. 36 (12), 1737-1746.

Clarke, G., Eubanks, D., Reid, E., Kelleher, C., O'Connor, E., DeBar, L.L., Lynch, F., Nunley, S., Gullion, C., 2005. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J. Med. Internet Res. 7 (2), e16.

Corrigan, P.W., 2006. Impact of consumer-operated services on empowerment and recovery of people with psychiatric disabilities. Psychiatr. Serv. 57 (10), 1493-1496.

Craig, T.K., Garety, P., Power, P., Rahaman, N., Colbert, S., Fornells Ambrojo, M., Dunn, G., 2004. The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ 329 (7474), 1067.

Crumlish, N., Whitty, P., Clarke, M., Browne, S., Kamali, M., Gervin, M., McTigue, O., Kinsella, A., Waddington, J.L., Larkin, C., O'Callaghan, E., 2009. Beyond the critical period: longitudinal study of 8-year outcome in first-episode non-affective psychosis. Br. J. Psychiatry 194 (1), 18-24.

Csikszentmihalyi, M., 1990. Finding Flow: the Psychology of Engagement with Everyday Life. Harper and Row, New York.

Davidson, L., Chinman, M., Sells, D., Rowe, M., 2006. Peer support among adults with serious mental illness: a report from the field. Schizophr. Bull. 32 (3), 443-450.

Depp, C.A., Mausbach, B., Granholm, E., Cardenas, V., Ben-Zeev, D., Patterson, T.L., Lebowitz, B.D., Jeste, D.V., 2010. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J. Nerv. Ment. Dis. 198 (10), 715-721.

Edwards, J., Elkins, K., M, H., Harrigan, S.M., Donovan, K., Athanasopoulos, O., McGorry, P.D., 2006. Randomised controlled trial of a cannabis-focused intervention for young people with first-episode psychosis. Acta Psychiatr. Scand. 114, 109-117.

Ewing, S., Thomas, J., Schiessl, J., 2008. CCI Digital Futures Report: the Internet in Australia. ARC Centre of Excellence for Creative Industries and Innovation Institute for Social Research. Swinburne University of Technology, Melbourne.

Eysenbach, G., 2005. The law of attrition. J. Med. Internet Res. 7 (1), e11.

First, M.B., Gibbon, M., Spitzer, R.L., Williams, J.B.W., 1996. Structured clinical interview for DSM-IV Axis I DisordersPatient Version (SCID-I/P). Biometrics Research Department, New York State Psychiatric Institute, New York, NY.

Friis, S., 2010. Early specialised treatment for first-episode psychosis: does it make a difference? Br. J. Psychiatry 196 (5), 339-340.

Gafoor, R., Nitsch, D., McCrone, P., Craig, T.K., Garety, P.A., Power, P., McGuire, P., 2010. Effect of early intervention on 5-year outcome in non-affective psychosis. Br. J. Psychiatry 196 (5),372-376.

Gleeson, J.F., Cotton, S.M., Alvarez-Jimenez, M., Wade, D., Gee, D., Crisp, K., Pearce, T., Newman, B., Spiliotacopoulos, D., Castle, D., McGorry, P.D., 2009. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients. J. Clin. Psychiatry 70 (4), 477-486.

Gleeson, J.F., Cotton, S.M., Alvarez-Jimenez, M., Wade, D., Gee, D., Crisp, K., Pearce, T., Spiliotacopoulos, D., Newman, B., McGorry, P.D., 2011. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients: outcome at 30-month follow-up. Schizophr. Bull.

Gleeson, J.F., Lederman, R., Wadley, G., Bendall, S., McGorry, P.D., Alvarez-Jimenez, M., in press. Successful management of safety, confidentiality and privacy within a moderated online social therapy for young people recovering from first-episode psychosis. J. Med. Internet Res.

Goh, T.T., Huang, Y., 2009. Monitoring youth depression risk in Web 2.0. Vine. J. Inf. Knowl. Manag. Syst. 39 (3), 192-202.

Gonzalez-Blanch, C., Crespo-Facorro, B., Alvarez-Jimenez, M., Rodriguez-Sanchez, J.M., Pelayo-Teran, J.M., Perez-Iglesias, R., Vazquez-Barquero, J.L., 2007. Cognitive dimensions in first-episode schizophrenia spectrum disorders. J. Psychiatr. Res. 41 (11), 968-977.

Granholm, E., Ben-Zeev, D., Link, P.C., Bradshaw, K.R., Holden, J.L., 2012. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive textmessaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull. 38 (3), 414-425.

Hall, P.L., Tarrier, N., 2003. The cognitive-behavioural treatment of low self-esteem in psychotic patients: a pilot study. Behav. Res. Ther. 41 (3), 317-332.

Huang, Y.-P., Goh, T., Liew, C.L., 2007. Hunting Suicide Notes in Web 2.0Preliminary Findings, Ninth IEEE International Symposium on Multimedia Workshops.

Jackson, H., McGorry, P., Edwards, J., Hulbert, C., Henry, L., Harrigan, S., Dudgeon, P., Francey, S., Maude, D., Cocks, J., Killackey, E., Power, P., 2005. A controlled trial of cognitively oriented psychotherapy for early psychosis (COPE) with four-year follow-up readmission data. Psychol. Med. 35 (9), 1295-1306.

Jackson, H.J., McGorry, P.D., Killackey, E., Bendall, S., Allott, K., Dudgeon, P., Gleeson, J., Johnson, T., Harrigan, S., 2007. Acute-phase and 1-year follow-up results of a randomized controlled trial of CBT versus befriending for first-episode psychosis: the ACE project. Psychol. Med. 1-11.

Jones, R.B., Atkinson, J.M., Coia, D.A., Paterson, L., Morton, A.R., McKenna, K., Craig, N., Morrison, J., Gilmour, W.H., 2001. Randomised trial of personalised computer based information for patients with schizophrenia. BMJ 322 (7290), 835-840.

Killackey, E., Jackson, H.J., McGorry, P.D., 2008. Vocational intervention in first-episode psychosis: individual placement and support v. treatment as usual. Br. J. Psychiatry 193 (2), 114-120.

Kreyenbuhl, J., Nossel, I.R., Dixon, L.B., 2009. Disengagement from mental health treatment among individuals with schizophrenia and strategies for facilitating connections to care: a review of the literature. Schizophr. Bull. 35 (4), 696-703.

Lederman, R., Wadley, G., Gleeson, J.F., Spiteri-Staines, A., Alvarez-Jimenez, M., 2011. Supporting young people with psychosis in the community: an ICT-enabled relapse prevention tool. Proceedings of the Pacific Asia Conference on Information Systems (PACIS), Brisbane.

Lee Duckworth, A., Steen, T.A., Seligman, M.E., 2005. Positive psychology in clinical practice. Annu. Rev. Clin. Psychol. 1, 629-651.

Lieberman, M.A., Goldstein, B.A., 2006. Not all negative emotions are equal: the role of emotional expression in online support groups for women with breast cancer. Psychooncology 15 (2), 160-168.

Linszen, D., Dingemans, P., Lenior, M., 2001. Early intervention and a five year follow up in young adults with a short duration of untreated psychosis: ethical implications. Schizophr. Res. 51 (1), 55-61.

Lloyd, R., Bill, A., 2004. Australia Online: How Australians Are Using Computers And the Internet. 2001 Australian Census Analytic Program. Publication 2056.0. Australian Bureau of Statistics, Canberra.

Lukoff, D., Liberman, R.P., Nuechterlein, K.H., 1986. Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophr. Bull. 12 (4), 578-593.

McFarlane, W.R., 2002. Multifamily Groups in the Treatment of Severe Psychiatric Disorders. Guilford Press, New York.

McFarlane, W.R., Lukens, E., Link, B., Dushay, R., Deakins, S.A., Newmark, M., Dunne, E.J., Horen, B., Toran, J., 1995. Multiple-family groups and psychoeducation in the treatment of schizophrenia. Arch. Gen. Psychiatry 52 (8), 679-687.

McGorry, P.D., Yung, A.R., 2003. Early intervention in psychosis: an overdue reform. Aust. N. Z.J. Psychiatry 37 (4), 393-398.

McGorry, P.D., Edwards, J., Mihalopoulos, C., Harrigan, S.M., Jackson, H.J., 1996. EPPIC: an evolving system of early detection and optimal management. Schizophr. Bull. 22 (2), 305-326.

McGorry, P.D., Killackey, E., Yung, A., 2008. Early intervention in psychosis: concepts, evidence and future directions. World Psychiatry 7 (3), 148-156.

Melle, I., Larsen, T.K., Haahr, U., Friis, S., Johannessen, J.O., Opjordsmoen, S., Simonsen, E., Rund, B.R., Vaglum, P., McGlashan, T., 2004. Reducing the duration of untreated first-episode psychosis: effects on clinical presentation. Arch. Gen. Psychiatry 61 (2), 143-150.

Mohr, D.C., Cuijpers, P., Lehman, K., 2011. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J. Med. Internet Res. 13 (1), e30.

Mueser, K.T., Drake, R.E., 2005. How does a practice become evidenced-based? In: Drake, R.E., Merrens, M.R., Lynde, D.W. (Eds.), Evidence-Based Mental Health Practice: a Textbook. W.W. Norton Company Ltd, New York, pp. 217-241.

Norman, R.M., Malla, A.K., 2001. Duration of untreated psychosis: a critical examination of the concept and its importance. Psychol. Med. 31 (3), 381-400.

Norman, R.M., Malla, A.K., Manchanda, R., Harricharan, R., Takhar, J., Northcott, S., 2005. Social support and three-year symptom and admission outcomes for first episode psychosis. Schizophr. Res. 80 (2-3), 227-234.

Norman, R.M., Manchanda, R., Malla, A.K., Windell, D., Harricharan, R., Northcott, S., 2011. Symptom and functional outcomes for a 5 year early intervention program for psychoses. Schizophr. Res. 129 (2-3), 111-115.

OWASP, the Open Web Application Security Project, t. www.akamai.com.

Penn, D.L., Uzenoff, S.R., Perkins, D., Mueser, K.T., Hamer, R., Waldheter, E., Saade, S., Cook, L., 2010. A pilot investigation of the Graduated Recovery Intervention Program (GRIP) for first episode psychosis. Schizophr. Res. 125 (2-3), 247-256.

Petersen, L., Jeppesen, P., Thorup, A., Abel, M.B., Ohlenschlaeger, J., Christensen, T.O., Krarup, G., Jorgensen, P., Nordentoft, M., 2005. A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ 331 (7517), 602.

Rashid, T., Ostermann, R.F., 2009. Strength-based assessment in clinical practice. J. Clin. Psychol. 65 (5), 488-498.

Riessman, F., 1965. The helper-therapy' principle. Soc. Work. 10, 27-32.

Rotondi, A.J., Anderson, C.M., Haas, G.L., Eack, S.M., Spring, M.B., Ganguli, R., Newhill, C., Rosenstock, J., 2010. Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr. Serv. 61 (11), 1099-1105.

Ryle, A., Kerr, I.B., 2003. Cognitive analytic therapy. Br. J. Psychiatry 183, 79.

Schuler, D., Namioka, A., 1993. Participatory Design: Principles and Practices. Routledge, Taylor & Francis group, Hillsdale, New Jersey.

Seligman, M.E., Rashid, T., Parks, A.C., 2006. Posit. Psychother. Am. Psychol. 61 (8), 774-788.

Sharkey, S., Jones, R., Smithson, J., Hewis, E., Emmens, T., Ford, T., Owens, C., 2011. Ethical practice in internet research involving vulnerable people: lessons from a selfharm discussion forum study (SharpTalk). J. Med. Ethics 37 (12), 752-758.

Singh, S.P., 2010. Early intervention in psychosis. Br. J. Psychiatry 196 (5), 343-345.

Spaniel, F., Vohlidka, P., Hrdlicka, J., Kozeny, J., Novak, T., Motlova, L., Cermak, J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res. 98 (1-3), 312-317.

Upthegrove, R., Birchwood, M., Ross, K., Brunett, K., McCollum, R., Jones, L., 2010. The evolution of depression and suicidality in first episode psychosis. Acta Psychiatr. Scand. 122 (3), 211-218.

Valimaki, M., Anttila, M., Hatonen, H., Koivunen, M., Jakobsson, T., Pitkanen, A., Herrala, J., Kuosmanen, L., 2008. Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis. Inform. Health Soc. Care 33 (2), 113-123.

Walker, H., 2006. Computer-based education for patients with psychosis. Nurs. Stand. 20 (30), 49-56.

Williams, C., Whitfield, G., 2001. Written and computer-based self-help treatments for depression. Br. Med. Bull. 57, 133-144.

Xia, J., Merinder, L.B., Belgamwar, M.R., 2011. Psychoeducation for schizophrenia. Cochrane Database Syst. Rev. 6, CD002831.

Issues in Mental Health Nursing, 35:323-329, 2014 Copyright ©2014 Informa Healthcare USA, Inc.

ISSN: 0161-2840 print/ 1096-4673 online

DOI: 10.3109/01612840.2013.863412

informa

healthcare


Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


A Comparison of Telephone and Texting Interventions for Persons with Schizophrenia Spectrum Disorders

Lora Beebe, PhD, RN

University of Tennessee, College of Nursing, Knoxville, Tennessee, USA

Kathlene D. Smith, PhD, RN

Tennessee Wesleyan College, Fort Sanders Nursing Program, Knoxville, Tennessee, USA

Chad Phillips, BSN, RN

University of Tennessee, College of Nursing, Knoxville, Tennessee, USA


Problem-solving interventions are not routinely offered to persons with schizophrenia spectrum disorders (SSDs). Telephone calls and text messages are potential avenues to offer problem solving support. This study compared the effect of telephone calls only, text messages only, and both telephone calls and text messages on individuals’ symptoms and medication adherence. Thirty outpatient participants with SSDs were randomly assigned to weekly telephone calls, daily text messages, or both for three months. Participants received monthly in-home pill counts and symptom assessments. Repeated measures ANOVA was significant (F (4,26) =

Schizophrenia spectrum disorders (SSDs) comprise schizophrenia and schizoaffective disorder. The similarities between the disorders include diagnostic overlap (American Psychiatric Association, 2013), symptomatology (Kopelow-icz, Ventura, Liberman, & Mintz, 2008), deficiencies in executive functioning (Premkumar et al., 2008), and associative learning deficits (Sacchetti, Galluzzo, Panariello, Parrinello, & Cappa, 2008). Another commonality is the robust response to problem-solving interventions exhibited by persons with these diseases (Hardeman, Harding, & Narasimhan, 2010; Zygmunt, Olfson, Boyer, & Mechanic, 2002). For the purpose of this project, “problem solving” is defined as the process described by D'Zurrila and Nezu (2006): (1) Identify the problem, (2) Generate/discuss solutions, (3) Select a solution, (4) Plan to implement the solution, and (5) Follow up on effectiveness of chosen solution.

Problem solving has long been among the most effective treatments to improve psychiatric medication adherence; the re-

Address correspondence to Lora Beebe, University of Tennessee,

College of Nursing, 1200 Volunteer Blvd., Knoxville, Tn 37996-4180

USA. E-mail: lbeebe1@utk.edu search evidence supporting this approach spans nearly 30 years (Falloon, McGill, Boyd, Pederson, 1987; Kopelowicz et al., 2012; Penn & Mueser, 1996). Problem solving is a component of many social skills programs for those with SSDs (Liberman et al., 1998) and also is associated with significant improvements in identifying problems and generating solutions when provided alone (Liberman, Eckman, & Marder, 2001).

In spite of their documented effectiveness and the recommendation by experts (Schooler et al., 1997; Zygmunt, Olfson, Boyer, & Mechanic, 2002), problem-solving interventions are not routinely offered to community-dwelling persons with SSDs (Lehman & Steinwachs, 2003). Two significant barriers to the adoption of face-to-face problem-solving programs are labor costs and poor intervention fidelity (Vallina-Fernandez et al., 2001). The most recently published survey examining problemsolving interventions in persons with SSDs concluded that less than half of the participants were receiving any type of problemsolving intervention (Lehman & Steinwachs, 2003). Telephone calls and text messages are potential avenues to meet the need for community-based problem-solving interventions that are accessible to patients at a low cost when compared to face-to-face intervention.

BACKGROUND

Five published studies have examined telephone or texting intervention for persons with schizophrenia spectrum disorders (SSDs). Telephone intervention of up to four months in length was provided via landline in four studies (Beebe, 2001; Beebe & Tian, 2004; Beebe et al., 2008; Montes, Maurino, Diez, & Saiz-Ruiz, 2011) and via cellular telephone in a five-month pilot investigation (Beebe et al., 2010).

Beebe (2001) compared weekly landline telephone intervention for three months following hospital discharge with treatment as usual in 40 persons with SSDs and concluded that this

323


Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


intervention delivery method is feasible and acceptable to patients. Beebe and Tian (2004) examined responses to landline telephone intervention over six weeks in 24 recently discharged persons with SSDs and reported that persons meeting the telephone intervention provider before calls began conversed significantly longer than controls not meeting the telephone intervention provider during weeks one, two, and three, and were twice as likely to make a feeling statement to the caller. In a follow up study, Beebe and colleagues (2008) compared medication adherence (via monthly, in-home pill count) in 29 persons with SSDs randomly assigned to weekly landline telephone intervention or treatment as usual for three months. Average psychiatric medication adherence was 80% for experimental participants (95% confidence interval [CI] = 67%-88%) and 60.1% (95% CI 47%-78%) for controls. A repeated measures ANOVA found a statistically significant mean effect for group (F (1,20) = 5.47, p = 0.0298), indicating significantly higher psychiatric medication adherence in those receiving the intervention. Montes and colleagues (2011) compared monthly landline telephone intervention for three months with usual care in 923 outpatient participants with schizophrenia. Adherence was rated by the treating psychiatrist and nurse (who were not blinded) using a four-item scale ranging from 1 (less than 20% of doses) to 4 (more than 80% of doses). Ratings were based upon participant self-report of missed doses and the presence or absence of acute symptoms. These authors (Montes et al., 2011) reported that a significantly higher percentage of patients in the telephone intervention group were classified in group 4 (more than 80% of doses) at the end of the study than controls. The single study examining the feasibility of cellular telephone intervention for persons with schizophrenia (Beebe et al., 2010) reported minor logistical problems, including participants’ difficulties retrieving messages, forgetting to take the telephone when leaving home, and forgetting to charge the telephone.

As a group, these studies demonstrate the feasibility and acceptability of telephone intervention for SSDs and provide preliminary evidence of telephone intervention’s effect upon medication adherence. The studies ranged from six weeks to five months in length, with an average retention rate of 84.75% and an average call completion rate of 73%. This body of work consistently demonstrates the feasibility and acceptability of this delivery method to patients, and documents statistically significant improvement in psychiatric medication adherence over usual care (Beebe et al., 2008).

We located a single published study exploring texting interventions for persons with SSDs. Granholm et al. (2012) provided daily text messages targeting medication adherence, socialization, and auditory hallucinations to 55 community-dwelling persons with SSDs (44 with schizophrenia) for 12 weeks. Text message interventions were based on cognitive behavioral therapy principles (Beck & Rector, 2000). Intervention goals were to provide reminders of health behaviors (e.g., taking prescribed medications); to encourage participants to question unhelpful beliefs, and to assist participants in creating personalized coping plans. The sample consisted mostly of Caucasian (74%) males (69%) and 76% of participants completed the 12-week study. Non-completers scored significantly lower on self-reported living skills and verbal IQ and had higher negative symptom scores than completers. Completers and non-completers did not differ on age, education, positive symptom scores, or depression. Selfreported medication adherence significantly improved among persons living independently, but not among persons living in supported environments. With a text response rate of 86%, Granholm and colleagues (2012) concluded that text messaging interventions are feasible and effective in SSDs, and recommended further investigation in more diverse samples

The purpose of this study was to compare the effect of telephone call intervention only, text messaging intervention only, and both telephone call and text message intervention, over three months, on symptoms and medication adherence in individuals with SSDs. We hypothesized that participants assigned to receive both telephone calls and texting intervention would have higher medication adherence and lower symptom scores than participants receiving only telephone calls or only the texting interventions.

METHODS

Participants and Recruitment

This study enrolled 30 outpatients with SSDs who were receiving care at a community mental health center (CMHC) in the Southeastern United States. The CMHC is a regional, not-for-profit integrated system providing outpatient services to 650+ patients with SSDs. Participants were contacted by the Principal Investigator (PI) and invited to participate in the study while they were at the CMMC for regularly scheduled treatment appointments. Inclusion criteria were: age between 21-68 years, receiving outpatient care at CMHC, chart diagnosis of schizophrenia (any subtype) or schizoaffective disorder using criteria described in Diagnostic and Statistical Manual for Mental Disorders (American Psychiatric Association, 2013), and English speaking. Exclusion criteria were: chart documentation of mental retardation or developmental delay, hearing loss prohibiting telephone communication, or vision or dexterity problems prohibiting texting.

After signing informed consent and providing demographic data, participants were randomly assigned (using a table of random numbers) to one of three groups: daily text message intervention by the PI, weekly telephone call intervention by a trained nurse other than the PI, or both daily text messages from the PI and weekly telephone calls by a trained nurse other than the PI for three months.

Interventions

Telephone Call Intervention

The weekly telephone calls consisted of an intervention developed by the first author and known as Telephone Intervention

Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


TABLE 1

Weekly TIPS Call Topics

Topic

missed any doses at all?

(Symptoms specific to each participant were collected at baseline.)

week that you’ve found uncomfortable?

Problem Solving for Schizophrenia (TIPS). TIPS is a manu-alized intervention that is used to provide weekly support to outpatients with SSDs, and has been described in detail elsewhere (Beebe, 2005). TIPS addresses the following common difficulties in community dwelling persons with SSDs: taking medication, attending appointments, coping with symptoms, abstaining from alcohol and other drugs, and getting along with others. Two open-ended items provide opportunities to address questions or other problems. See Table 1 for a list of TIPS topics (manual available from first author). Trained nurses guide persons with SSDs through the problem-solving process for any difficulties identified, generating solutions, choosing a solution, and following up on the effectiveness of the solution at the next contact. For example, a client reporting problems with medication adherence due to forgetting doses would be assisted to identify several solutions such as making a reminder sign, identifying a family member or caregiver to provide reminders, or tying medication to some other routine, such as brushing one’s teeth. Clients select the solution they wish to implement; the solution and results are reviewed at the next TIPS contact.

Texting Intervention

Based upon the text messaging multiple choice format used by Granholm et al. (2012), we adapted the TIPS telephone protocol for delivery via text message format with multiple choice responses. Each participant received a daily text message topic from the PI for three months. The text message procedure manual (available from first author) provides a format consisting of six distinct message topics with multiple choice responses; messages repeat every six days. See Table 2 for an overview of text message topics. The PI responded to identified difficulties with participant-specific problem solving.

Combined Telephone Intervention and Texting Intervention

Participants in the combined telephone and texting intervention group received weekly phone calls and daily text messages as described above for the three months of the study.

TABLE 2

Daily Text Message Topics

Topic

medications. Did you take your medications today? You can say Yes, Some, I don’t want to, or I forgot.

appointments. Do you know when your next appointment is scheduled? You can say Yes, No, or I’m not sure.

Have you been bothered by any [specific symptom here]? You can say Not at all, A little, or A lot.

Have you had any cravings for alcohol or other drugs that you’ve found uncomfortable? You can say Not at all, A little, or A lot.

have you been getting along with others? You can say Very well, Pretty well, or Not so well.

you have questions about.

appreciate you! Have a great day!

Measures

Pill Counts

The PI initiated telephone contact to schedule monthly inhome pill counts. Participants were requested to produce all medications dispensed in pill form to be counted. A measure of adherence was generated by dividing the number of pills missing from the bottle(s) by the number of pills prescribed within the time period of the prescription. Adherence percentage was calculated separately for each oral medication in pill form using the following formula:

/ Number of pills missing from the bottle \ Number of pills prescribed since prescription refill date

As an example, consider this hypothetical participant who filled a medication prescription on July 1 for 30 pills of 100 mg Seroquel, with the instruction to take one by mouth at bedtime. At the pill count visit on July 31 at 3 pm, the blinded research assistant found 25 pills missing from the bottle when a total of 30 pills had been prescribed since the refill date. 25/30 x 100 2500/30 = 83.3% adherence for Seroquel over the specified time period for this participant.

For participants prescribed depot medication, adherence for the depot medication was calculated (via record review) as a percentage of injections received of the total number of injections prescribed during each month. It is the clinical practice at

Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


the recruitment site to record depot prescriptions in three-month intervals. As an example, consider this hypothetical participant who was prescribed to receive 50mg of Haldol Decoanate intramuscularly every two weeks from June 1 through August 31. Injection clinics take place once a week on the same day, so the three injection days for July were Monday July 1, 15, and 29. If the participant presented for the two injections scheduled on July 15 and 29, their adherence percentage for Haldol Depot would be 2/3, or 66.6%. All participants receiving depot medication also were receiving oral psychiatric and/or nonpsychiatric medications; procedures for their in-home pill count and oral adherence calculations were identical to those not prescribed depot medication.

If more than one psychiatric or nonpsychiatric medication was prescribed, an overall adherence percentage was calculated by averaging the percentage adherence of all medications within that category (both oral and depot). Overall adherence was calculated as the mean adherence for all medications in a group (psychiatric or nonpsychiatric). For this study, we defined psychiatric medications as typical and atypical antipsychotics, antidepressants, anti-anxiety medications, mood stabilizers, antiparkinsonian medications, and hypnotics. All other medications were defined as nonpsychiatric medications. Procedures for inhome pill counts were identical to those used in our prior work (Beebe et al, 2010).

Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1988)

The BPRS is an 18-item clinician rated scale addressing a broad range of psychopathology. Scores ranged from 0-108 with higher scores indicating greater disturbance. Weighted Kappa coefficients ranged from 0.52-0.90 for individual items with a mean of 0.72 for all items (Gabbard et al., 1987). The measure has good internal consistency with Cronbach’s alphas of 0.81 and 0.91 for positive and negative symptoms, respectively (Nicholson, Chapman, & Neufeld, 1995) and good concurrent validity (Faustman, 1994; Guy, 1976; Hedlund & Vieweg, 1980). The BPRS takes about 30 minutes to administer and score. After reviewing the interview scale/scoring guidelines, the blinded research assistant (RA) and PI practiced scoring the BPRS with simulated patients. Scores were recorded independently; mean inter-rater reliability was 0.89.

Procedures

Potential participants were approached while at the recruitment site for their regularly scheduled treatment appointments. HIPAA law and the Notice of Privacy Practices, signed by all patients at the recruitment site, allow disclosure of Protected Health Information (PHI) for research, authorizing the initial case reviews and communications required to identify potential participants. The PI approached 37 persons meeting the study criteria regarding participation; seven persons declined. Three persons cited a lack of confidence using a cellular telephone as a reason for declining, two reported feeling unable to participate due to their symptom level, and two objected to the time commitment. The 30 persons agreeing were randomly assigned (using a table of random numbers) to weekly TIPS only, daily text messages only, or both weekly TIPS and daily text messages for three months.

Sociodemographic characteristics, living arrangements, educational level, prescribed medications, and BPRS scores were collected at baseline by the PI in a private office at the CMHC immediately following the granting of informed consent but before group assignment. Thus, the PI was blinded as to group assignment when conducting the baseline BPRS. All study participants were provided a basic cellular telephone with unlimited calling and texting for three months. The PI provided daily text messages to participants randomly assigned to receive them and documented responses by hand. A trained nurse (a TIPS provider in two prior studies, Beebe et al., 2008 and Beebe et al., 2010) provided TIPS interventions and documented participant responses by hand. All participants received monthly in-home pill counts and BPRS evaluations for three months by the RA (different than the TIPS nurse). Following each monthly assessment, participants received a $10 Wal-Mart gift card.

RESULTS

The sample consisted of 19 females and 11 males with a mean age of 48.7 years (range: 23-64, SD = 11.6). The sample was equally divided between Caucasian and African American participants and most participants lived alone (n = 16). The most commonly prescribed medications were oral atypical antipsychotics (n = 20) and anti-parkinsonian agents (n = 17). Fifteen participants were prescribed at least one nonpsychiatric medication, most commonly anti-hypertensives (n = 14) (see Table 3).

Twenty-eight participants completed the three-month study. One participant’s telephone was exposed to water on day three; unfortunately, study funds did not allow purchase of replacement telephones. One participant was hospitalized on day 54 and was unable to complete the study due to deteriorating physical condition. Both participants not completing the study were assigned to the text only group. Hence, ten participants in the TIPS only group, ten participants in the TIPS plus text group, and eight participants in the text only group completed the study.

BPRS Scores

There were no statistically significant baseline differences among the three groups on any demographic variables examined; however a one way ANOVA revealed a statistically significant difference in baseline BPRS scores (F (2,27) = 6.1, p = 0.007); the mean baseline BPRS score of those assigned to receive TIPS only was 50.1, the mean baseline BPRS score of the text only group was 41.3, and the mean baseline BPRS score of the TIPS plus text group was 37.7 (see Table 4).

Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


TABLE 3

Characteristics of Participants with SSDs (N = 30)

Characteristic

n (%)

Diagnosis

Schizoaffective

20 (66.7)

Schizophrenia

10 (33.3)

Sex

Female

19 (63.3)

Male

11 (36.7)

Race

Caucasian

15 (50)

African American

15 (50)

Living Arrangement

Alone

16 (53.3)

With family

11 (36.7)

With caregiver

3(10)

Prescribed Medications

Oral atypicals

20 (66.7)

Oral typicals

4 (13.3)

Depot typicals

12 (40)

Depot atypicals

5 (16.7)

Antidepressants

14 (46.7)

Mood stabilizers

12 (40)

Anti-anxiety

11 (36.7)

Anti-parkinson

17 (56.7)

Hypnotics

4 (13.3)

Other

15 (50)

’Includes medications prescribed for physical illnesses, most commonly anti-hypertensives (n = 14) and calcium channel blockers (n = 12).


TABLE 4

BPRS Scores by Group of Persons with SSDs at Baseline and 1, 2, and 3 Months after Intervention (N = 28-30)

Group

Baseline Range Mean (SD)

Month 1

n Range Mean (SD)

Month 2 n

Range

Mean (SD)

Month 3 Range Mean (SD)

TIPS Plus

Text

10

10

10

10

31-48

25-60

24-58

24-67

37.7 (6.1)

38.2(11.9)

36.8 (10.9)

35.8 (12.8)

Text Only

10

10

8

8

35-51

25-50

30-60

31-64

41.3 (5.2)

31.8 (9.7)

46.5(10.9)

44.5(11.6)

TIPS Only

10

10

10

10

38-76

27-57

31-61

25-65

50.1 (11.7)

38.5 (9.1)

47.6 (9.5)

41.7(12.4)

’Missing data due to drop out.


Results of Hypothesis Testing

Three 2 x 3 ANOVA models with one independent measure (Group) and one repeated measure (Time) were calculated to determine differences in psychiatric medication adherence, nonpsychiatric medication adherence, and symptom levels based on intervention type, with percentage adherence to psychiatric medications as the dependent variable in the first model, percentage adherence to nonpsychiatric medications as the dependent variable in the second model, and BPRS scores as the dependent variable in the third model. In our ANOVA models we examined three different variance and covariance structures (compound symmetry, unstructured, and Huynh-Felt) while controlling for differences in baseline BPRS scores. The results of the three analyses were similar; results of the unstructured analysis are reported in this article. These analyses yielded a statistically significant main effect for group (F (4,26) =

There was no significant Group x Time interaction for psychiatric, F (4,26) = 1.24, p = 0.31) or nonpsychiatric, F (4,26) = 0.53, p = 0.71) medication adherence. Nevertheless, our findings were in the predicted direction for both psychiatric and nonpsychiatric medication adherence: Mean psychiatric adherence scores for the TIPS plus text group were higher than both the TIPS only (by an average of 5.3%) and the text only groups (by an average of 13%) at each of the three postintervention measurement points (see Table 5). Mean nonpsychiatric medication adherence scores for the TIPS plus text group were higher (by an average of 11.9%) than the text only group at two of the three post-intervention measurement points, and higher than the TIPS only group (by an average of 14.9%) at all three post-intervention measurement points (see Table 6). A post hoc analysis revealed that, based upon the sample size examined, the power to examine psychiatric and nonpsychiatric medication adherence was 34% and 25%, respectively.

Additional Analyses

We used Chi square to examine relationships among ordinal variables. Where at least one cell of the chi square had an expected count of less than five, Fischer's exact tests were computed. There were no significant associations among the ordinal variables examined with the exception of living arrangement—significantly more participants in the TIPS only group lived with family (n = 7) than those in the text only (n = 3) or TIPS plus text (n = 1) groups. (Fischer's exact = 9.77, p = 0.02).

Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


TABLE 5

Psychiatric Medication Adherence Scores (Percent of Doses Taken) of Persons with SSDs 1, 2, and 3 Months after Intervention (N = 28-30)

Group

Month 1

n Range Mean (SD)

Month 2

n

Range

Mean (SD)

Month 3

n Range Mean (SD)

TIPS Plus Text

10

10

9*

35-100

64-100

20-100

84.2 (22.4)

87.5(13.0)

81.1 (25.5)

Text Only

10

8*

8*

0-100

13-100

33-100

72 (33.7)

70.1 (33.2)

71.5 (26.6)

TIPS Only

10

10

10

33-100

41-100

60-100

72 (20.1)

83.9(18.0)

80.9(16.3)

*Missing data due to drop out.


DISCUSSION

To our knowledge this is the first study comparing text and telephone intervention for persons with SSDs. Our findings are congruent with the small body of literature documenting increased medication adherence with telephone intervention. Our prior work using identical pill count procedures documented 80% psychiatric medication adherence over three months in those receiving TIPS (Beebe et al., 2008); similarly, the average psychiatric medication adherence over this three-month study was 84.3% in the TIPS plus text group and 78.9% in the TIPS only group. Likewise, Montes et al. (2011) reported that significantly more participants receiving telephone intervention took at least 80% of their prescribed medication doses (ratings based upon self-report and symptom ratings).

TABLE 6

Nonpsychiatric Medication Adherence Scores (Percent of Doses Taken) of Persons with SSDs 1, 2, and 3 Months after Intervention (N = 28)

Group

Month 1

n Range Mean (SD)

Month 2

n

Range

Mean (SD)

Month 3

n Range Mean (SD)

TIPS Plus Text

7

8

8

56-100

57-100

48-100

82.1 (19.9)

86.6 (7.6)

76.9 (20.9)

Text Only

6

5*

5*

30-100

28-100

49-100

82.2 (26.5)

69.4 (33.9)

70.2 (27.2)

TIPS Only

7

6

6

40-94

9-83

40-100

73.1 (21.8)

58.5 (27.2)

69.3 (24.9)

*Missing data due to drop out.


Our finding of nonsignificant differences in medication adherence between TIPS only, text only, and TIPS plus text participants requires further elaboration. A possible explanation is that differences in the groups were small due to the lack of a control condition of usual care. In our prior study, which included a treatment as usual control group, those assigned to the control condition had an average psychiatric medication adherence of 60.1% over three months (Beebe et al., 2008). Furthermore, eight clients in our sample (40%) were receiving services from the Program for Assertive Community Treatment (PACT), a 24hour, intensive treatment delivery system for high service users. PACT clients have their medication delivered daily by treatment staff, resulting in very high medication adherence, and requiring a larger sample to detect the small adherence increases observed. Psychiatric medication adherence in the eight participants assigned to PACT was 96.7% in month 1, 96.2% in month 2, and 98% in month 3.

We located no other published work examining symptom levels of persons with SSDs receiving telephone intervention. Our finding of significantly lower BPRS scores, even when baseline scores were controlled for, provides preliminary evidence for the efficacy of TIPS in symptom reduction, particularly when combined with daily text messages. Mean BPRS scores of the TIPS plus text group were lower than the text only group (by an average of 9.2 points) at two of three postintervention measurement points. Mean BPRS scores of the TIPS plus text group were lower than the TIPS only group (by an average of 5.7 points) at all three post-intervention measurement points.

LIMITATIONS

The use of a small, convenience sample recruited from a single community mental health center limits the generalizability of our findings. The smaller than expected increases in medication adherence resulted in power less than 35% to detect significant differences in adherence with the current sample size, introducing the possibility of Type II error. Finally, although pill counts are a commonly used proxy measure for medication adherence, pill counts do not guarantee that missing medications were actually ingested (Beebe et al., 2008).

The inclusion of participants receiving depot medications could have increased our adherence rates, since depot formulations are associated with approximately 10% greater adherence than oral psychiatric medications (Zoler, 2012). Fifteen participants were receiving depot medications in addition to other oral psychiatric medications: seven in the TIPS only group; five in the text only group; and three in the TIPS plus text group. In spite of this limitation, the inclusion of persons receiving depot medications results in a final sample that may more accurately represent the target population for the TIPS intervention.

Issues Ment Health Nurs Downloaded from informahealthcare.com by Kungliga Tekniska Hogskolan on 03/06/15 For personal use only.


CONCLUSION

This study provides preliminary evidence for the superiority of telephone intervention plus text messages over either telephone or text intervention alone in symptom reduction and medication adherence increases in persons with SSDs. Our findings indicate that adding text messages to the TIPS intervention provides small gains in both symptom reductions and increased adherence to psychiatric and nonpsychiatric medications. Further, it appears that text messages are a feasible adjunct intervention in SSDs. More research with larger, more diverse samples is needed to refine text messaging procedures, to identify subgroups for which this intervention might be optimally useful, and to examine the intervention’s effects over longer time periods. Our follow-up study will examine the correlations between in-home pill count and serum medication levels over nine months.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

REFERENCES

American Psychiatric Association. (2013). Diagnostic andstatistical manual of mental disorders (5th ed.). Washington, DC: Author.

Beck, A. T., & Rector, N. A. (2000). Cognitive therapy of schizophrenia: A new therapy for the new millennium. American Journal of Psychotherapy, 54, 291-300.

Beebe, L. H. (2001). Community nursing support for persons with schizophrenia. Archives of Psychiatric Nursing, 15(5), 214-222.

Beebe, L. H. (2005). Telephone Intervention-Problem Solving (TIPS) for Persons with schizophrenia. Directions in Psychiatric Nursing, 11(9), 103-112

Beebe, L. H., Smith, K., Bennett, C. et al. (2010). Keeping in touch: Cellular telephone use in persons with schizophrenia spectrum disorders. Journal of Psychosocial Nursing and Mental Health Services, 48(4), 32-37.

Beebe, L. H., Smith, K., Crye, C. et al. (2008). Telenursing intervention increases psychiatric medication adherence in schizophrenia outpatients. Journal of the American Psychiatric Nurses Association, 14(3), 217-224.

Beebe, L. H., & Tian, L. (2004). TIPS: Telephone Intervention-Problem Solving for persons with schizophrenia. Issues in Mental Health Nursing, 25(3), 317-329.

D’Zurrila, T. D., & Nezu. A. M. (2006). Problem-solving therapy: A positive approach to clinical intervention (3rd ed.). New York, NY: Springer.

Falloon, I. R., McGill, C. W., Boyd, J. L., & Pederson, J. (1987). Family management in the prevention of morbidity of schizophrenia: Social outcome of a two-year longitudinal study. Psychological Medicine, 17(1), 59-66.

Faustman, W. O. (1994). Brief psychiatric rating scale. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 371-401). Hillsdale, NJ: Erlbaum.

Gabbard, G. O., Kennedy, L. L., Deering, C. D. et al. (1987). Inter-rater reliability in the use of the brief psychiatric rating scale. Bulletin of the Menninger Clinic, 51, 519-531.

Granholm, E., Ben-Zeev, D., Link, P. C., Bradshaw, K. R., & Holden, J. L. (2012). Mobile Assessment and Treatment for Schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization and auditoryhallucinations. SchizophreniaBulletin, 38(3), 414-425.

Guy, W. (1976). Assessment manual for psychopharmacology revised. DHEW pub # ADM 76-338. Rockville MD, US Department of Health Education and Welfare, Public Health Service, Alcohol, Drug Abuse and Mental Health Administration, National Institute of Mental Health Psychopharmacology Research Branch, Division of Extramural Research Programs.

Hardeman, S. M., Harding, R. K., & Narasimhan, M. (2010). Simplifying adherence in schizophrenia. Psychiatric Services, 61(4), 405-408.

Hedlund, J. L., & Vieweg, B. W. (1980). The brief psychiatric rating scale (BPRS): A comprehensive review. Journal of Operational Psychiatry, 11, 48-65.

Kopelowicz, A., Ventura, J., Liberman, R. P., & Mintz, J. (2008). Consistency of brief psychiatric rating scale factor structure across a broad spectrum of schizophrenia patients. Psychopathology, 41(2), 77-84.

Kopelowicz, A., Zarate, R., Wallace, C. J., Liberman, R. P., Lopez, S. R., & Mintz, J. (2012). The ability of multifamily groups to improve treatment adherence in Mexican Americans with schizophrenia. Archives of General Psychiatry, 69(3), 265-273.

Lehman, A. F., & Steinwachs, D. M. (2003). Evidence-based psychosocial treatment practices in schizophrenia: Lessons from the patient outcomes research team (PORT) project. Journal of the American Academy of Psychoanalytic and Dynamic Psychiatry, 31(1), 141-154.

Liberman, R. P., Eckman, T. A., & Marder, S. R. (2001). Rehab rounds: Training in social problem solving among persons with schizophrenia. Psychiatric Services, 52(1), 31-33.

Liberman, R. P., Wallace, C. J., Blackwell, G., Kopelowicz, A., Vaccaro, J. V., & Mintz, J. (1998). Skills training versus psychosocial occupational therapy for persons with persistent schizophrenia. American Journal of Psychiatry155(8), 1087-1091.

Montes, J. M., Maurino, J., Diez, T., & Saiz-Ruiz, J. (2011). Factors associated with the effectiveness of a telephone based nursing strategy for enhancing medication adherence in schizophrenia. Clinical Practice in Epidemiology and Mental Health, 7, 117-119.

Nicholson, I. R., Chapman, J. E., & Neufeld, R. W. J. (1995). Variability in BPRS definitions of positive and negative symptoms. Schizophrenia Research, 17, 177-185.

Overall, J. E., & Gorham, D. R. (1988). The Brief Psychiatric Rating Scale (BPRS) recent developments in ascertainment and scaling. Psychopharma-cological Bulletin, 24, 97-99.

Penn, D. L., & Mueser, K. T. (1996). Research update on the psychosocial treatment of schizophrenia. American Journal of Psychiatry, 153(5), 607-617.

Premkumar, P., Cooke, M. A., Fannon, D., Peters, E., Michel, T. M., Aasen, I., Murray, R. M., Kuipers, E., & Kumari, V. (2008). Misattribution bias of threat-related facial expressions is related to a longer duration of illness and poor executive function in schizophrenia and schizoaffective disorder. European Psychiatry, 23(1), 14-19.

Sacchetti, E., Galluzzo, A., Panariello, A., Parrinello, G., & Cappa, S. F. (2008). Self-ordered pointing and visual conditioning associated learning tasks in drug free schizophrenia spectrum disorder patients. BMC Psychiatry, 23(8), 6.

Schooler, N. R., Keith, S. J., Severe, J. B. et al. (1997). Relapse and rehospitalization during maintenance treatment of schizophrenia. The effects of dose reduction and family treatment. Archives of General Psychiatry, 54(5), 453-463.

Vallina-Fernandez, O., Lemos-Giraldez, S., Roder, V. et al. (2001). Rehab rounds: An integrated psychological treatment program for schizophrenia. Psychiatric Services, 52(9), 1165-1167.

Zoler, M. (2012, May 23). Depot injection boosts antipsychotic compliance in schizophrenia. Clinical Psychiatry News. Retrieved from http://www.clinicalpsychiatrynews.com

Zygmunt, A., Olfson, M., Boyer, C. A., & Mechanic, D. (2002). Interventions to improve medication adherence in schizophrenia. American Journal of Psychiatry, 159(10), 1653-1664.


Contents lists available at ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres


Pilot randomised controlled trial of a brief coping-focused intervention for hearing voices blended with smartphone-based ecological momentary assessment and intervention (SAVVy): Feasibility, acceptability and preliminary clinical outcomes

Imogen H. Bell a, 2, Susan L. Rossell a,b, John Farhall c,d, Mark Hayward e,f, Michelle H. Lim a, Sarah F. Fielding-Smith e,f, Neil Thomas a

a Centre for Mental Health, Swinburne University of Technology, Australia

b Department of Psychiatry, St. Vincent's Hospital, Australia

c Department of Psychology and Counselling, La Trobe University, Australia

d NorthWestern Mental Health, Melbourne Health, Australia

e Sussex Partnership NHS Foundation Trust, UK

f School of Psychology, University of Sussex, UK

ARTICLE INFO


ABSTRACT


Article history:

Received 12 January 2019

Received in revised form 25 June 2019

Accepted 10 October 2019 Available online xxx


Keywords:

Digital technology

Experience sampling methodology

Treatment

Psychosis

Auditory hallucinations

Blended therapy


Background: Voice-hearing experiences can be distressing and impairing, and existing psychological treatments show modest effectiveness. Ecological momentary assessment and intervention (EMA/I) are two promising approaches which may be used as digital tools to support and enhance existing psychological therapies. The aim of this study was to investigate the potential clinical utility of smartphonebased EMA/I in a blended, coping focused therapy for voice-hearing experiences. Method: This pilot RCT focused on feasibility, acceptability and preliminary estimations of efficacy. Thirty-four participants with persisting and distressing voices were randomised to receive the four-session intervention along-side treatment-as-usual (TAU) or TAU-only. Results: Findings supported the feasibility and acceptability of the approach, with good engagement and satisfaction rates, and clinical outcomes showed the intervention holds promise for improving coping, overall severity of voices and to some degree their negative impact. Conclusion: This is the first examination of the use of EMA/I in a blended therapy for psychotic experiences, with findings suggesting these technologies show promise as clinical tools.

© 2019 Elsevier B.V. All rights reserved.

1. Introduction

Psychological interventions, particularly cognitive behavioural therapy for psychosis (CBTp; Farhall and Thomas, 2013; Turkington et al., 2008), are recommended as a core component of treatment for hearing voices, or auditory verbal hallucinations (Galletly et al., 2016; NICE, 2010). However, effects sizes of CBTp are modest (Jauhar et al., 2014; van der Gaag et al., 2014), access is generally limited, and the treatment is costly and complex to deliver (Berry and Haddock, 2008; Schizophrenia Commission, 2012; Haddock et al., 2014; Ince et al., 2016).

Smartphones applications (apps) have unique capabilities that may provide novel and innovative ways to improve the effectiveness and reach of psychological treatments for psychosis (Bakker et al., 2016; Price et al., 2014; Proudfoot, 2013; Thomas et al., 2019; Treisman et al., 2016). Ownership of smartphones is wide spread in psychosis populations (Firth et al., 2015) and research has shown that these technologies can assist in illness selfmanagement, reduce symptoms and their impact, minimise relapse, and promote physical health (Alvarez-Jimenez et al., 2014Bell et al., 2017; Firth and Torous, 2015). Further, individuals with psychosis appear interested in using them regularly for their mental health (Bucci et al., 2018b; Gay et al., 2016; Torous et al.,

Ecological momentary interventions (EMIs; Heron and Smyth,

Ecological momentary assessment (EMA; Shiffman et al., 2008) is a related approach which uses smartphone apps to deliver questionnaires in daily life at repeated intervals across several days. Greater reliability and ecological validity are afforded through measurement of phenomenon in the moment, in natural environments (Shiffman et al., 2008; Stone et al., 2007). Further, repeated measurement allows for the examination of temporal relationships between variables (Ebner-Priemer and Trull, 2009). Findings have supported the feasibility and reliability of using EMA in psychosis populations (Brenner and Ben-Zeev, 2014; Granholm et al., 2008Palmier-Claus et al., 2012), however little research has investigated its use in clinical treatment (Bell et al., 2017; McDevitt-Murphy, Luciano and Zakarian, 2018).

EMA and EMI (EMA/I) may be useful within blended therapies for psychosis, involving the combined used of digital technologies with standard face-to-face therapies. In other clinical populations, studies have shown that these approaches may lead to more potent interventions (Erbe et al., 2017). It may be possible to adapt existing evidence-based treatments to incorporate technologies such as EMA/I to support therapeutic components. In the context of voicehearing experiences, one such candidate therapy is coping strategy enhancement (CSE; Tarrier, 1992; Tarrier et al., 1990).

CSE is an idiographic, CBT-based psychological therapy aiming to improve coping with psychotic symptoms (Tarrier, 1992; Tarrier et al., 1990). Functional analysis is used to identify antecedents and responses to symptoms, which then informs the identification and subsequent implementation of individualised coping strategies. Trials have supported the clinical benefit of CSE approaches (Tarrier et al., 1993,1998; Yusupoff and Tarrier, 1996), including in a brief format over four sessions targeting voices specifically (Hayward et al., 2018; Paulik et al., 2018).

Conceivably, such an approach may be enhanced by incorporating EMA to assist in initial functional analysis, providing data on variation in voices and related variables, and EMI prompts of individualised coping strategies may promote more consistent use of these in daily life. This highly novel application of EMA/I was the subject of this research, which aimed to examine the feasibility, acceptability and estimated clinical effects of a brief intervention which blended EMA/I with standard face-to-face therapy to improve coping with hearing voices [Smartphone-Assisted coping focused interVention for Voices (SAVVy)]. The development of the intervention and a case illustration is reported in Bell et al. (2018a).

2. Methods

Reporting followed CONSORT guidelines (supplementary) and ethical approval was provided by the Alfred Hospital Ethics Committee (project 440/16). The trial was prospectively registered (ACTRN12617000348358) and the study protocol was published before recruitment ended (Bell et al., 2018b).

2.1. Study design

A single-blind, parallel group, pilot RCT with a 1:1 allocation ratio to the SAVVy intervention plus treatment-as-usual (TAU) or TAU alone. TAU typically included standard care provided by a clinical team, including medication and case management. Trained researchers blind to treatment allocation completed assessments at pre-randomisation and approximately 8 weeks following randomisation. An independent researcher randomly allocated participants to groups used minimisation procedure using QMinim online software. Minimisation was used to balance continuous vs non-continuous voices across groups [Psychotic Symptom Rating Scales-Auditory Hallucinations (PSYRATS-AH; Haddock et al., 1999) item 1 score <3 versus 4], a variable which may be influenced by EMA/I.

2.2. Participants

Thirty-four adult participants were recruited through referrals to a specialist Voices Clinic and wider advertising to clinical services and consumer groups, between March 2017 and January 2018. The sample size was based on published guidelines (Julious, 2005Sim and Lewis, 2012; Teare et al., 2014) and is consistent with similar recent pilot RCTs with this population (Bucci et al., 2018Hazell et al., 2018). Eligibility criteria were: (1) over the age of 18 years; (2) proficient English language; (3) experiencing current, frequent (4 + times per week, or if less, lasting at least 1 h at a time) and distressing (score 1 + on amount of distress item of PSYRATS-AH (Haddock et al., 1999) voices for at least six months; (4) comfortable using a smartphone or willing to learn. Exclusion criteria were (1) unable to provide informed consent; (2) intellectual disability (estimated IQ< 70, measured by the Wechsler Test of Adult Reading (WTAR; Wechsler, 2001); (3) initiation of a new antipsychotic medication within the previous 8 weeks; (4) voices solely substance-related; (5) distress or agitation displayed during baseline assessment; and (6) requiring active crisis management.

2.3. Measures

Baseline measures included basic demographic and clinical information (e.g. medication dosages); use and familiarity with technology; Mini-International Neuropsychiatric Interview (Sheehan et al., 1998) for DSM-5 mental disorders; Structured Clinical Interview for DSM-5 (First et al., 2015) for borderline personality disorder diagnosis; Scale for the Assessment of Negative Symptoms (SANS; Andreasen, 1989); and Wechsler Test of Adult Reading (WTAR; Wechsler, 2001) to estimate intellectual ability.

Feasibility was the primary outcome measure, focusing on: completion rates of the EMA questionnaires (completers defined as having completed over 33% of the total number of EMA questionnaires), proportion of participants for whom EMA-based feedback summaries were produced; proportion of EMI reminders viewed; trial uptake and attrition; and fidelity to the intervention protocol (proportion of therapy checklist items endorsed by therapists as completed).

Acceptability was measured using a feedback questionnaire designed for the study. Participants completed the Credibility and Expectancy Questionnaire (CEQ; Devilly and Borkovec, 2000) after the informed consent procedure, of which the Credibility subscale was used to measure pre-conceived perceptions of the intervention credibility. Treatment group participants completed the Working Alliance Inventory Short Revised (WAI-SR; Hatcher and Gillaspy, 2006) in relation to their rapport with the therapist.

The primary clinical outcome was PSYRATS-AH total score (Haddock et al., 1999). Secondary clinical outcomes included Depression, Anxiety and Stress Scale-21 total score (DASS-21; Lovibond and Lovibond, 1995) and the Subjective Experiences of Psychosis Scale Negative Impact Subscale total score (SEPS; Gillian Haddock et al., 2011). Process measures included two visual analogue scales (VAS) to assess (1) confidence in coping with voices

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx                                                              3

day-to-day, and (2) awareness of patterns in voices, and two multiple choice items measuring the frequency of use of coping strategies and the number of strategies used.

2.4. Intervention

Details of the development and a case study illustrating the delivery of the intervention are reported elsewhere (Bell et al., 2018a). A depiction of the intervention procedure is displayed in Fig. 1. The intervention was split into two phases involving initial assessment and EMA monitoring for functional analysis, which informed the second phase involving identifying and implementing individualised coping strategies which were supported by personalised EMI reminders in daily life. An existing app called MovisensXS was used for the purpose of the trial. Participants were lent a smartphone if they did not already have a compatible Android phone (necessary to run the app).

Following the first session involving an introduction and training in how to use the smartphone app, participants completed six days of EMA monitoring involving the completion of a 39-item questionnaire (supplementary 2), ten times per day. EMA items measured common antecedents to voices (e.g. mood, anxiety), voice-related variables (i.e. intensity, distress, impact), and coping responses to the voices (e.g. listening to music, arguing with the voices). EMA items were determined based on an iterative approach involving reviews of the EMA literature on psychotic symptoms and coping with voices, feedback from lived experience consultants, researchers and clinicians in the field, and an initial pilot study (Bell et al., 2018a). This EMA period facilitated selfmonitoring of voices, and provided data that was then statistically analysed using time-lagged multiple regressions to identify variables associated with fluctuations in voice intensity (see Bell et al., 2018a, 2018b). Summary statistics of questionnaires completion rates, voice intensity and distress, and coping strategies used were also computed. In line with recommendations in the literature, this feedback analysis was conducted if participants completed at least 33% of the total number of EMA surveys (Delespaul, 1995; Palmier-Claus et al., 2011). A simple, lay-person summary of this analysis was provided to participants in the second session, which was discussed in an exploratory manner to inform functional analysis of the voices. This functional analysis was used to identify alternative responses to the voices which may interrupt problematic maintenance cycles associated with their activity and improve overall coping. Individual coping strategies were worded as short sentences by the participant and programmed into the app. Participants then received five personalised EMI prompts per day for the following ten days after session two and were able to view their reminders on-demand. Eight evening EMA questions were used to monitor changes in the voices and helpfulness of the coping strategies, with feedback of this information then reviewed in session three and coping strategies could be updated if needed. This was followed by a further ten-day EMI period and evening EMA questions, with the final session involving a review and ending of the intervention.

2.5. Analysis

Feasibility and acceptability results are reported descriptively. Clinical outcomes were analysed on an intention-to-treat basis using all available data. Missing cases were treated as missing at random with a small number of individual missing data points (<5%) imputed using an expectation-maximization method and multiple imputation used for the three missing outcome cases (pooled aggregate of 50 iterations; Enders, 2001; Tabachnick and


Table 1

Demographic and Clinical Characteristics of each group.

Variable

SAVVy + TAU (n = 17)

TAU (n = 17)

Age M(SD)

39.12 (10.64)

42.59 (10.64)

Gender (%Female)

64.7%

47.1%

Marital status n(%)

Single

70.6%

64.7%

Divorced

5.9%

17.6%

Defacto

5.9%

11.8%

Married

5.9%

0%

Separated

11.8%

0%

Country of birth n(%) Australia

76.5%

82.4%

Central/South America

0%

5.9%

UK or Europe

5.9%

0%

New Zealand

5.9%

0%

India or Asia Subcontinent

5.9%

5.9%

South East Asia

0%

5.9%

Middle East

5.9%

0%

Ethnicity n(%) Australian

70.6%

94.1%

New Zealander

5.9%

0%

British or Irish

11.6%

0%

Greek

5.9%

5.9%

Other

5.9%

0%

Primary language n(%)

English

100%

100%

Level of education n(%)

Year 10 or less

11.8%

23.5%

Year 11/12

41.2%

17.6%

Diploma

17.6%

23.5%

Bachelor's degree

17.6%

23.5%

Post graduate diploma/Graduate Certificate

0%

5.9%

Current employment n(%)

Employed full time

5.9%

11.8%

Employed part time

0%

11.8%

Casually employed

11.8%

11.8%

Unemployed

41.2%

47.1%

Student

29.4%

5.9%

Volunteer

0%

11.8%

Home duties

11.8%

0%

Primary diagnosis n(%)

Bipolar Disorder w psychotic feat.

11.8%

17.7%

Major Depression w psychotic feat.

0%

5.9%

Schizoaffective Disorder

29.4%

47.1%

Schizophrenia

52.9%

23.5%

Schizophreniform

0%

5.9%

Unspecified Schizophrenia Spectrum Disorder

5.9%

0%

SANS M(SD)

8.82 (6.36)*

17.58 (11.21)*

Chlorpromazine equivalence M(SD)

519.06 (419.97)

296.73 (385.29)

WTAR M(SD)

103.35 (14.45)

99.18 (8.61)

Owns a smartphone n(%)

94.1%

70.6%

Has internet access n(%)

97%

100%

Use of internet (median)

More than once per day (58.8%)

More than once per day (47.1%)

Confidence using apps M(SD)a

5.76 (1.79)

4.06 (2.28)

Note: *significant difference between groups; aConfidence using apps was measured on a 7-point Likert type scale from 1(very unconfident) to 7(very confident).


Fidell, 2007). Clinical outcomes are presented as pooled means and standard deviations, with Hedges' g formula used to calculate standardised effect sizes and associated 95% confidence intervals (Durlak, 2009). Effect sizes for all variables were coded so that positive values reflected changes favouring the treatment group. In line with our planned analysis, analysis of covariance (ANCOVA) was used to test for differences between treatment and control group scores on all clinical outcome and process measures at the outcome timepoint, controlling for baseline scores. For the two ordinal measures of frequency and number of coping strategies used, Wilcoxon Sign Ranks Test were used to compare pre-post changes within each group. All outcome analyses were conducted using IBM SPSS Statistics Version 25 (SPSS Inc, 2017). EMA and EMI feasibility statistics and EMA-derived feedback analysis within the intervention were conducted using Stata Version 14.1 (StataCorp, 2015).

3. Results

3.1. Sample characteristics

Demographic and clinical characteristics of both groups are provided in Table 1. Group comparisons revealed significantly higher SANS scores in the control compared to treatment group (F(1,32) = 7.86, p < .01), with no other differences identified.

3.2. Feasibility

Fig. 2 displays the CONSORT flow diagram. Of those screened for eligibility, there was a 34% uptake into the trial. Data were available for 31 (91%) participants at post-treatment, with 3 participants lost to follow-up (2 control, 1 treatment) and no withdrawals. Of the 17 participants in the treatment group, 13 completed all four sessions.


Two stated they could no longer find time for participation (one attending three sessions and the other one session), and the two remaining participants discontinued sessions due to worsening in mental health precluding their ability to attend appointments (one attending three sessions and the other one session). These reasons were judged as factors external to the trial and unrelated to the intervention. Group comparisons did not reveal any significant differences between those who did and did not drop out on any demographic or clinical variables, or confidence using smartphones (p > .05). Ninety percent of therapy checklist items were endorsed as completed across the four sessions.

Two participants discontinued therapy during the EMA monitoring period, and another experienced a technological issue with the smartphone alerts, and was therefore excluded from EMA/I completion analysis. Across the remaining 14 participants, the average completion rate of the daytime EMA questionnaires was 72%. One-hundred percent of participants reached the minimum 33% completion rate criteria necessary to produce the EMA-derived feedback. Unplanned Pearson's correlation analyses did not reveal any significant relationships (p > .05) between completion rates and confidence in using smartphone apps, demographic or clinical variables, or clinical outcome variables. Completion rates of evening EMA questionnaires across the intervention was 74%. Scheduled EMI reminders were viewed on average 2.5 times per day, and

3.3. Acceptability

Table 2 displays the average responses to each item of the satisfaction questionnaire. Overall, responses reflected good satisfaction across all elements of the intervention, with 100% of treatment group participants agreeing that they would recommend it to other people who hear voices. Open feedback was minimal, but largely positive, with all verbatim responses displayed in Table 3. The average WAI-SR item score (range 15) was 4.33 (SD = 0.55), suggesting that participants developed positive working alliances with the therapist. The average CEQ scores of the 3-item Credibility subscale (range 19) were 7.5 (S = 1.56) for the perceived logic of the therapy, 6.68 (SD = 1.53) for the perceived success of the therapy, and 6.56 (SD = 2.21) for the confidence in recommending the therapy to others. Mean scores on the CEQ and WAI-SR were similar to those reported in other trials in analogous populations (Gaudiano et al., 2015; Webb et al., 2013; White et al., 2011).

3.4. Clinical outcomes

Scores on all clinical outcome and process measures for both groups at baseline and outcome timepoints are displayed in Table 4. Moderate effect sizes favouring the treatment group were observed for PSYRATS-AH total score with a trend towards significant difference between groups (F(1,31) = 3.00, p = .09). Small, nonsignificant effects favouring the treatment group for the SEPS negative impact subscale (F(1,31) = 0.55, p = .46), and very small, non-significant effects favouring the control group on DASS-21 scores (F(1,31) = 1.87, p = .18), were observed. A very large effect favouring the treatment group was observed for the VAS coping with voices item and differences between groups were significant (F(1,31) = 23.59, p < .001). Similarly, the VAS awareness of patterns in voices item was also significantly different between groups (F(1,31) = 5.40, p < .05), with a moderate effect size. Analyses with significant group differences were run again with SANS as an additional covariate. Group differences for the VAS coping with

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx

Table 2

Means and standard deviations of item responses to the satisfaction survey.

Item and range

M

SD

1(strongly disagree) to 5(strongly agree)

1. Overall, the smartphone app was easy to use

4.43

0.94

2. Monitoring my voice/s using the app helped me to understand more about these experiences

4.21

0.70

3. The questions in the app were easy to understand

4.50

0.65

4. The coping strategy reminders were useful to help me cope with my voice/s

4.29

0.61

5. It was useful to discuss the smartphone feedback in therapy

4.57

0.51

6. The feedback about the coping strategies was useful

4.50

0.65

7. The feedback about patterns in my voice/s was helpful in understanding my experiences

4.43

0.65

8. I found the therapy helpful overall

4.50

0.65

9. I would recommend this intervention to other people with voice-hearing experiences

4.64

0.50

1(not often enough) to 5(too often)

10. The number of beeps from the app were ...

3.36

0.84

11. The number of coping strategy reminders from the app were ...

3.21

0.58

1(not enough) to 5(too many)

12. The number of sessions were .

3.21

0.58

Table 3

Verbatim open feedback from participants.

Negative                                                                         Positive

Just the problems with the phone and the app in the second stage. Became a bit Thanks to this study I now realise I can have a better and more happy, stress-free and tedious.a                                                                      peaceful life if I can take a good look objectively at the experience of the voices and

The initial number of prompts in the first week was a little annoying.              work out some constructive ways of dealing with them.

It required a lot of time and effort. Which is okay but its been a busy time of the year It helped me to control my voices and to make me feel better about myself.

for the first time in many years and I found it hard to focus on everything and keep It helped me to make a connection between the voices and my own thoughts and up.b                                                                           feelings. It reminded me to take care of myself.

Discussing what I am talking, getting confused at times about my past.            Identifying triggers and patterns was helpful in terms of understanding my voices,

When it beeped at inappropriate times, middle of writing an email or at church. how to cope with them, and the discussion I had with the therapist.

I could talk about the voices and they could understand how I was feeling. Collaboratively exploring new ways to understand my negative self-sabotaging voice and how it can and does affect me in daily activities.

Good to discuss what I experience with someone who understands.

I tended to be able to notice when the voices were getting worse or better. A good reminder about time passing, to eat, do something.

Detecting patterns in the voices.

There was nothing I didn't like, it was all good and I had the support to cope with the voices.

a participant experienced a technical issue with the smartphone app.

b participant dropped out due to conflicting study and work commitments during the trial.

Table 4

Summary statistics and effects on clinical measures for each group.

SAVVy + TAU (n

= 17)

TAU (n = 17)

Hedges g

95% Confidence Intervals

Measure

Baseline

Outcome

Baseline

Outcome

Mean (SD)

Mean (SD)

Mean (SD)

Mean (SD)

PSYRATS Total

28.47 (4.87)

25.89 (6.37)

28.76 (4.41)

29.47 (6.45)

0.55

-0.14, 1.23

SEPS Negative Impact Subscale

91.24 (28.03)

83.08 (26.00)

95.88 (23.44)

90.59 (22.53)

0.30

-0.37, 0.98

DASS-21 Overall

58.24 (32.77)

65.61 (28.30)

65.88 (32.14)

60.11 (31.88)

-0.18

-0.85, 0.50

Coping items

1. Confidence in coping

56.00 (30.83)

65.77 (21.90)

55.76 (23.02)

33.24 (22.05)

1.45***

0.69, 2.20

2. Understanding of voices

53.53 (33.40)

72.63 (23.03)

53.18 (27.37)

56.39 (28.35)

0.61**

-0.07, 1.30

Median (IQR)

Median (IQR)

Median (IQR)

Median (IQR)

3. Frequency of coping strategy use*

3 (2-4)

4 (3-4)

3 (2-4)

2 (2-4)

4. Number of coping strategies*

2 (2-3)

3 (3-4)

1 (2-3)

2 (2-3)

Note. Means and standard deviations incorporate pooled imputations; ***p < .001; **p < .01; *p < .05.

voices item remained significant ((F(1,30) = 27.03, p < .001), but was no longer significant for the VAS awareness of patterns in voices item (F(1,30) = 2.90, p = .09). Wilcoxon Signed Ranks tests indicated a marginally significant pre-post increase in the number of coping strategies (Z = -1.89, p = .06), but not the frequency of their use (Z = -0.37, p = .71), in the treatment group. There were no changes in the control group on the number of coping strategies (Z = -0.36, p = .72) nor frequency of their use (Z = -0.24, p = .81). Unplanned Pearson's correlations were run to examine whether confidence in using smartphone apps was related to any scores on

clinical or process measures within the treatment group at the outcome time point, with none reaching significance (p > .05). Sensitivity analyses were conducted which revealed minimal differences in the above analyses between imputed and non-imputed data sets (Thabane et al., 2013).

Fig. 3 displays box plots of PSYRATS-AH change scores in each group. There was an outlier in the treatment group representing a pre-post increase (i.e. worsening) in PSYRATS-AH total score. A review of assessment recordings and a follow-up qualitative interview indicated this was likely to be due to an external life event occurring around the time of the outcome assessment. A sensitivity analysis excluding this individual resulted in the effect on PSYRATS-AH becoming significant [(F(1,30) = 6.36, p < .05; Hedges g = 0.61, 95% CI = -0.07,1.33], with other results remaining the same.

3.5. Adverse events

Two events classed as serious adverse events (hospital admissions) according to the Australian National Health & Medical Research Council ([NHMRC], 2007) were reportedone from the treatment group and other from control. A review concluded these were unrelated to the trial or intervention.

4. Discussion

The findings of this study support the feasibility and acceptability of a brief coping-focused intervention for distressing voicehearing experiences which blended standard face-to-face psychological therapy with EMA/I between session. Completion rates of the EMA questionnaires were high, leading to the production of EMA-derived feedback in all attempted cases, and there was good engagement with both prompted and user-initiated EMI coping reminders. Despite minor technological issues, feedback regarding different aspects of the intervention was largely positive. These findings extend digital mental health research in psychosis by demonstrating that smartphone technologies can support standard face-to-face therapies; otherwise known as blended therapy (Erbe et al., 2017).

Research has demonstrated that EMI can support independent self-management of psychosis (Bell et al., 2017; Bucci et al., 2018Schlosser et al., 2018). Our findings suggest that personalised EMI reminders of tailored self-management strategies determined during therapy may support the generalisation of these strategies into daily life. Participants were engaged with these EMI reminders and feedback suggested they were helpful. It is conceivable that personalised EMI reminders may be a simple and useful technology for other psychological treatment approaches. Future developments may involve more streamlined programming of EMI content and the use of context-aware systems to determine the timing and nature of tailored EMI prompts (Bakker et al., 2016Burns et al., 2011; Price et al., 2014; Proudfoot, 2013).

A significant novelty of this study is the analysis of within-person EMA data to inform clinical formulation in psychological therapy. No other study has examined this application of EMA in psychosis, despite considerable interest (Ebner-Priemer and Trull, 2009; Firth and Torous, 2015; McDevitt-Murphy et al., 2018Myin-Germeys et al., 2016; Oorschot et al., 2012; Trull and Ebner-Priemer, 2009). Although one case experienced a technological issue whereby the alerts were not consistently received, the overall high completion rate of the EMA questionnaires led to the production of EMA-derived feedback in all attempted cases. This, alongside predominantly positive feedback from participants, supports the feasibility and acceptability of this approach. The high level of engagement with both EMA and EMI components are consistent with other similar studies (Ben-Zeev et al., 2014; Berkel et al., 2017; Bucci et al., 2018; Firth and Torous, 2015; Kumar et al., 2018), although recent findings suggest naturalistic engagement with apps may be lower (Torous et al., 2017). Notably, one participant who dropped out commented on the effort involved in the intervention, suggesting this approach may be more difficult for those with limited time. Whilst we did not find evidence of a relationship between the characteristics of the sample and engagement with, nor effects of, the intervention, further research in this area would be beneficial to identify what works best and for whom (Michie et al., 2017; Ritterband et al., 2006).

It is hoped that this study, being the first of its kind, spurs further research exploring different statistical and methodological approaches to conducting within-person analysis of EMA data for clinical purposes. We used regression analyses and summary statistics, however other statistical approaches, such as network analysis (Bringmann et al., 2013), machine learning (Burns et al.,

Although only a pilot trial, post-intervention effects were in a direction favouring the treatment group for the primary clinical outcome of overall severity of voices, and to a lesser extent the secondary outcome of negative impact of the voices, but not emotional distress. More proximal, process, measures indicated statistically significant improvements favouring the treatment group in confidence in coping with voices, awareness of factors influencing voices, and close to significant increase in the number (but not frequency) of coping strategies used. These findings suggest this intervention holds promise for reducing the overall severity of voices and their negative impact, possibly occurring via the process of improved coping and understanding of voices. The very large effects observed on the measure of confidence in coping with voices provides proof-of-concept evidence for the mechanisms of the intervention. As these processes were targeted directly by EMA/I, this suggests the technology component was of benefit. These effects appear consistent with prior trials of CSE-based interventions (Hayward et al., 2018; Paulik et al., 2018; Tarrier et al., 1993, 1998; Yusupoff and Tarrier, 1996), however direct comparisons are limited due to variations in methodology. Notably, one participant within the treatment group showed a worsening in the primary outcome measure which may have lowered our conservative estimate of the average effect size. Whilst our investigation suggested an external life event was the main contributing factor,

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx


8

this highlights caution in ensuring adverse events are considered in any larger scale trialling, an area possibly neglected in CBT for psychosis research (Morrison, 2018).

The following limitations should be recognised. Firstly, it is unclear if the effects of the intervention were maintained as there was no follow-up time-point. Secondly, the intervention contained multiple components, limiting inferences regarding specific therapeutic mechanisms. A future trial should carefully consider an active comparison group (e.g., Bucci et al., 2018), including the recently highlighted digital placebo effect (Torous and Firth, 2016). Dismantling studies to isolate the active ingredients of digital interventions may assist in refining the features which yield maximum benefits, whilst improving our understanding of the mechanisms which drive them (Collins et al., 2007; Michie et al., 2017). Thirdly, the small sample size limits generalisability and clearly a fully powered trial is needed to determine clinical efficacy. Fourth, group difference in negative symptoms at baseline were observed, however it is noted that subsequent analyses controlling for this variable resulted in only minor changes.

Conclusion

The current study demonstrates the clinical potential of EMA/I as tools within blended therapies for psychotic experiences. This justifies potential further development of a purpose-built mobile app with evaluation in a full scale RCT to determine efficacy, and potentially investigations within other psychological treatment approaches and clinical populations.

Declaration of competing interest

The authors declare there are no conflicts of interest.

Acknowledgments

The authors thank the participants, lived experience consultants, and researchers Professor Denny Meyer, Dr Wei Lin Toh, Dr Rachel Brand, Ms Inge Gnatt and Ms Louise Moncur.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.schres.2019.10.026.

Contributors

IHB and NT conceptualised the intervention. IHB led the design of the protocol, with contributions from all other authors. IHB conducted the trial, with oversight by NT. SLR and NT assisted with the analysis, which was conducted by IHB. IHB wrote the first draft of this manuscript, with contributions from all other authors. All authors contributed to and have approved the final manuscript.

Funding

This research was supported by the Australian Government Research Training Program Scholarship and the Barbara Dicker Brain Sciences Foundation Grant Scheme.

References

Alvarez-Jimenez, M., Alcazar-Corcoles, M., Gonzalez-Blanch, C., Bendall, S., McGorry, P., Gleeson, J., 2014. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr. Res. 156 (1), 96-106.

Andreasen, N.C., 1989. Scale for the assessment of negative symptoms (SANS). Br. J. Psychiatry 155 (Suppl 7), 53-58.

Bakker, D., Kazantzis, N., Rickwood, D., Rickard, N., 2016. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Mental Health 3 (1).

Barnett, I., Torous, J., Staples, P., Keshavan, M., Onnela, J.P., 2018. Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data. J. Am. Med. Inform. Assoc. 25 (12), 1669-1674.

Bell, I.H., Lim, M.H., Rossell, S.L., Thomas, N., 2017. Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. Psychiatr. Serv. 68 (11), 1172-1181.

Bell, I.H., Fielding-Smith, S.F., Hayward, M., Rossell, S.L., Lim, M.H., Farhall, J., Thomas, N., 2018a. Smartphone-based ecological momentary assessment and intervention in a blended coping-focused therapy for distressing voices: development and case illustration. Internet Interventions 14, 18-25. https:// doi.org/10.1016/j.invent.2018.11.001.

Bell, I.H., Fielding-Smith, S.F., Hayward, M., Rossell, S.L., Lim, M.H., Farhall, J., Thomas, N., 2018b. Smartphone-based ecological momentary assessment and intervention in a coping-focused intervention for hearing voices (SAVVy): study protocol for a pilot randomised controlled trial. Trials 19 (1), 262. https:// doi.org/10.1186/s13063-018-2607-6.

Ben-Zeev, D., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., Mueser, K.T., 2014. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr. Bull. 40 (6), 1244-1253.

Berkel, N.V., Ferreira, D., Kostakos, V., 2017. The experience sampling method on mobile devices. ACM Comput. Surv. 50 (6), 93.

Berry, K., Haddock, G., 2008. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychol. Psychother. Theory Res. Pract. 81 (4), 419-436.

Brenner, C., Ben-Zeev, D., 2014. Affective forecasting in schizophrenia: comparing predictions to real-time Ecological Momentary Assessment (EMA) ratings. Psychiatr. Rehabil. J. 37 (4), 316.

Bringmann, L.F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., Peeters, F., et al., 2013. A network approach to psychopathology: new insights into clinical longitudinal data. PLoS One 8 (4), e60188.

Bucci, S., Barrowclough, C., Ainsworth, J., Machin, M., Morris, R., Berry, K., et al., 2018a. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophrenia Bulletin, p. sby032.

Bucci, S., Morris, R., Berry, K., Berry, N., Haddock, G., Barrowclough, C., et al., 2018b. Early psychosis service user views on digital technology: qualitative analysis. JMIR Mental Health 5 (4), e10091.

Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., Mohr, D.C., 2011. Harnessing context sensing to develop a mobile intervention for depression. JMIR 13 (3), e55.

Collins, L.M., Murphy, S.A., Strecher, V., 2007. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am. J. Prev. Med. 32 (5), S112-S118.

Delespaul, P., 1995. Assessing Schizophrenia in Daily Life - the Experience Sampling Method. Maastricht. IPSER Foundation, Netherlands.

Devilly, G.J., Borkovec, T.D., 2000. Psychometric properties of the credibility/ex-pectancy questionnaire. J. Behav. Ther. Exp. Psychiatry 31 (2), 73-86.

Dixon, L.B., Holoshitz, Y., Nossel, I., 2016. Treatment engagement of individuals experiencing mental illness: review and update. World Psychiatry 15 (1), 13-20.

Durlak, J.A., 2009. How to select, calculate, and interpret effect sizes. J. Pediatr. Psychol. 34 (9), 917-928.

Ebner-Priemer, U.W., Trull, T.J., 2009. Ambulatory assessment: an innovative and promising approach for clinical psychology. Eur. Psychol. 14 (2), 109-119.

Enders, C.K., 2001. A primer on maximum likelihood algorithms available for use with missing data. Struct. Equ. Model. 8 (1), 128-141.

Erbe, D., Eichert, H.-C., Riper, H., Ebert, D.D., 2017. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. JMIR 19 (9).

Farhall, J., Thomas, N., 2013. Cognitive and behavioural therapies for psychosis. Aust. N. Z. J. Psychiatr. 47 (6), 508-511. https://doi.org/10.1177/0004867413483370.

Fioravanti, M., Bianchi, V., Cinti, M.E., 2012. Cognitive deficits in schizophrenia: an updated metanalysis of the scientific evidence. BMC Psychiatry 12 (1), 64.

First, M.B., Williams, J.B.W., Karg, R.S., Spitzer, R.L., 2015. Structured Clinical Interview for DSM-5Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). American Psychiatric Association, Arlington, VA.

Firth, J., Torous, J., 2015. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth and uHealth 3 (4), e102.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2015. Mobile phone ownership and endorsement of mHealthamong people with psychosis: a meta-analysis of cross-sectional studies. Schizophr. Bull. 42 (2), 448-455.

Fisher, A.J., 2015. Toward a dynamic model of psychological assessment: implications for personalized care. J. Consult. Clin. Psychol. 83 (4), 825.

Galletly, C., Castle, D., Dark, F., Humberstone, V., Jablensky, A., Killackey, E., et al., 2016. Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for the management of schizophrenia and related disorders. Aust. N. Z. J. Psychiatr. 50 (5), 410-472.

Gaudiano, B.A., Busch, A.M., Wenze, S.J., Nowlan, K., Epstein-Lubow, G., Miller, I.W., 2015. Acceptance-based behavior therapy for depression with psychosis: results from a pilot feasibility randomized controlled trial. J. Psychiatr. Pract. 21 (5), 320.

Gay, K., Torous, J., Joseph, A., Pandya, A., Duckworth, K., 2016. Digital technology use

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx


among individuals with schizophrenia: results of an online survey. JMIR Mental Health 3 (2).

Granholm, E., Loh, C., Swendsen, J., 2008. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophr. Bull. 34 (3), 507-514.

Haddock, G., McCarron, J., Tarrier, N., Faragher, E., 1999. Scales to measure dimensions of hallucinations and delusions: the psychotic symptom rating scales (PSYRATS). Psychol. Med. 29 (04), 879-889.

Haddock, G., Wood, L., Watts, R., Dunn, G., Morrison, A.P., Price, J., 2011. The Subjective Experiences of Psychosis Scale (SEPS): psychometric evaluation of a scale to assess outcome in psychosis. Schizophr. Res. 133 (1), 244-249.

Haddock, G., Eisner, E., Boone, C., Davies, G., Coogan, C., Barrowclough, C., 2014. An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia. J. Ment. Health 23 (4), 162-165.

Hatcher, R.L., Gillaspy, J.A., 2006. Development and validation of a revised short version of the Working Alliance Inventory. Psychother. Res. 16 (1), 12-25.

Hayward, M., Edgecumbe, R., Jones, A.M., Berry, C., Strauss, C., 2018. Brief coping strategy enhancement for distressing voices: an evaluation in routine clinical practice. Behav. Cognit. Psychother. 46 (2), 226-237.

Hazell, C.M., Hayward, M., Cavanagh, K., Jones, A.M., Strauss, C., 2018. Guided selfhelp cognitive-behaviour Intervention for VoicEs (GiVE): results from a pilot randomised controlled trial in a transdiagnostic sample. Schizophr. Res. 195, 441-447.

Heron, K., Smyth, J., 2010. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br. J. Health Psychol. 15 (1), 1-39.

Ince, P., Haddock, G., Tai, S., 2016. A systematic review of the implementation of recommended psychological interventions for schizophrenia: rates, barriers, and improvement strategies. Psychol. Psychother. Theory Res. Pract. 89 (3), 324-350.

Jauhar, S., McKenna, P., Radua, J., Fung, E., Salvador, R., Laws, K., 2014. Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. Br. J. Psychiatry 204 (1), 20-29.

Johansen, R., Hestad, K., Iversen, V.C., Agartz, I., Sundet, K., Andreassen, O.A., Melle, I., 2011. Cognitive and clinical factors are associated with service engagement in early-phase schizophrenia spectrum disorders. J. Nerv. Ment. Dis. 199 (3), 176-182.

Julious, S.A., 2005. Sample size of 12 per group rule of thumb for a pilot study. Pharm. Stat.: The Journal ofApplied Statistics in the Pharmaceutical Industry 4 (4), 287-291.

Kumar, D., Tully, L.M., Iosif, A.-M., Zakskorn, L.N., Nye, K.E., Zia, A., Niendam, T.A., 2018. A mobile health platform for clinical monitoring in early psychosis: implementation in community-based outpatient early psychosis care. JMIR Mental Health 5 (1).

Lovibond, P.F., Lovibond, S.H., 1995. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav. Res. Ther. 33 (3), 335-343.

McDevitt-Murphy, M.E., Luciano, M.T., Zakarian, R.J., 2018. Use of ecological momentary assessment and intervention in treatment with adults. Focus 16 (4), 370-375.

National Health and Medical Research Council, 2007. National statement on ethical conduct in human research 2007 (Updated May 2015). Retrieved from. https:// nhmrc.gov.au/about-us/publications/national-statement-ethical-conduct-human-research-2007.

Michie, S., Yardley, L., West, R., Patrick, K., Greaves, F., 2017. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. JMIR 19 (6).

Morrison, A.P., 2018. Should people with psychosis be supported in choosing cognitive therapy as an alternative to antipsychotic medication: a commentary on current evidence. Schizophrenia Research.

MovisensXS. Retrieved from. https://xs.movisens.com/.

Myin-Germeys, I., Klippel, A., Steinhart, H., Reininghaus, U., 2016. Ecological momentary interventions in psychiatry. Curr. Opin. Psychiatr. 29 (4), 258-263.

National Institute for Health and Care Excellence [NICE], 2010. Core Interventions in the Treatment and Mangement of Schizophrenia in Adults in Primary and Secondary Care, updated edition. UK The British Psychological Society and The Royal College of Psychiatrists, London.

Oorschot, M., Lataster, T., Thewissen, V., Wichers, M., Myin-Germeys, I., 2012. Mobile assessment in schizophrenia: a data-driven momentary approach. Schiz-ophr. Bull. 38 (3), 405-413.

Palmier-Claus, J.E., Ainsworth, J., Machin, M., Barrowclough, C., Dunn, G., Barkus, E., et al., 2012. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry 12 (1), 172.

Palmier-Claus, J., Myin-Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P., et al., 2011. Experience sampling research in individuals with mental illness: reflections and guidance. Acta Psychiatr. Scand. 123 (1), 12-20.

Paulik, G., Jones, A.M., Hayward, M., 2018. Brief coping strategy enhancement for distressing voices: P redictors of engagement and outcome in routine clinical practice. Clin. Psychol. Psychother. 25 (5), 634-640.

Price, M., Yuen, E.K., Goetter, E.M., Herbert, J.D., Forman, E.M., Acierno, R., Ruggiero, K.J., 2014. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin. Psychol. Psychother. 21 (5), 427-436.

Proudfoot, J., 2013. The future is in our hands: the role of mobile phones in the

9 prevention and management of mental disorders. Aust. N. Z. J. Psychiatr. 47 (2), 111-113. https://doi.org/10.1177/0004867412471441.

QMinim. Retrieved from http://qminim.sourceforge.net.

Ritterband, L.M., Andersson, G., Christensen, H.M., Carlbring, P., Cuijpers, P., 2006. Directions for the international society for research on internet interventions (ISRII). JMIR 8 (3).

Schizophrenia Commission, T.S., 2012. The Abandoned Illness: a Report from the Schizophrenia Commission. Illness RM, London.

Schlosser, D.A., Campellone, T.R., Truong, B., Etter, K., Vergani, S., Komaiko, K., Vinogradov, S., 2018. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr. Bull. 44 (5), 1010-1020.

Sheehan, D.V., Lecrubier, Y., Sheehan, K.H., Amorim, P., Janavs, J., Weiller, E., et al., 1998. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.J. Clin. Psychiatry 59 (Suppl 20), 22-33.

Shiffman, S., Stone, A., Hufford, M., 2008. Ecological momentary assessment. Annu. Rev. Clin. Psychol. 4, 1-32.

Sim, J., Lewis, M., 2012. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J. Clin. Epidemiol. 65 (3), 301-308.

SPSS Inc, 2017. IBM SPSS Statistics for Windows (Version. IBM Corp, Armonk, NY, version 25.0.

StataCorp, 2015. Stata Statistical Software: Release 14. StataCorp LP, College Station, TX.

Stone, A., Shiffman, S., Atienza, A., Nebeling, L., 2007. The Science of Real-Time Data Capture: Self-Reports in Health Research. Oxford University Press.

Tabachnick, B.G., Fidell, L.S., 2007. Using Multivariate Statistics, fifth ed. Allyn & Bacon/Pearson Education, Boston, USA.

Tarrier, N., 1992. Management and modification of residual positive psychotic symptoms. In: Birchwood, M., Tarrier, N. (Eds.), Innovations in the Psychological Management of Schizophrenia: Assessment, Treatment and Services. Wiley Series in Clinical Psychology, pp. 147-169.

Tarrier, N., Harwood, S., Yusopoff, L., Beckett, R., Baker, A., 1990. Coping strategy enhancement (CSE): a method of treating residual schizophrenic symptoms. Behav. Psychother. 18 (04), 283-293.

Tarrier, N., Beckett, R., Harwood, S., Baker, A., Yusupoff, L., Ugarteburu, I., 1993. A trial of two cognitive-behavioural methods of treating drug-resistant residual psychotic symptoms in schizophrenic patients: I. Outcome. Br. J. Psychiatry 162 (4), 524-532.

Tarrier, N., Yusupoff, L., Kinney, C., McCarthy, E., Gledhill, A., Haddock, G., Morris, J., 1998. Randomised controlled trial of intensive cognitive behaviour therapy for patients with chronic schizophrenia. BMJ 317 (7154), 303-307.

Teare, M.D., Dimairo, M., Shephard, N., Hayman, A., Whitehead, A., Walters, S.J., 2014. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials 15 (1), 1.

Thabane, L., Mbuagbaw, L., Zhang, S., Samaan, Z., Marcucci, M., Ye, C., et al., 2013. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med. Res. Methodol. 13 (1), 92.

Thomas, N., Bless, J.J., Alderson-Day, B., Bell, I.H., Cella, M., Craig, T., et al., 2019. Potential applications of digital technology in assessment, treatment, and selfhelp for hallucinations. Schizophr. Bull. 45 (Suppl. ment_1), S32-S42.

Torous, J., Firth, J., 2016. The digital placebo effect: mobile mental health meets clinical psychiatry. The Lancet Psychiatry 3 (2), 100-102.

Torous, J., Friedman, R., Keshavan, M., 2014. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR mHealth and uHealth 2 (1), e2.

Torous, J., Staples, P., Slaters, L., Adams, J., Sandoval, L., Onnela, J., Keshavan, M., 2017. Characterizing smartphone engagement for schizophrenia: results of a naturalist mobile health study. Clin. Schizophrenia Relat. Psychoses. In-Press. [Epub ahead of print].

Treisman, G.J., Jayaram, G., Margolis, R.L., Pearlson, G.D., Schmidt, C.W., Mihelish, G.L., et al., 2016. Perspectives on the use of eHealth in the management of patients with schizophrenia. J. Nerv. Ment. Dis. 204 (8), 620.

Trull, T., Ebner-Priemer, U., 2009. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol. Assess. 21 (4), 457-462.

Turkington, D., Kingdon, D., Weiden, P.J., 2008. Cognitive behavior therapy for schizophrenia. Am. J. Psychiatry 6 (2), 257-266.

van der Gaag, M., Valmaggia, L.R., Smit, F., 2014. The effects of individually tailored formulation-based cognitive behavioural therapy in auditory hallucinations and delusions: a meta-analysis. Schizophr. Res. 156 (1), 30-37.

Webb, C.A., Kertz, S.J., Bigda-Peyton, J.S., Bjorgvinsson, T., 2013. The role of pretreatment outcome expectancies and cognitive-behavioral skills in symptom improvement in an acute psychiatric setting. J. Affect. Disord. 149 (1-3), 375-382.

Wechsler, D., 2001. Wechsler Test of Adult Reading (WTAR). The Psychological Corporation, San Antonio, TX.

White, R., Gumley, A., McTaggart, J., Rattrie, L., McConville, D., Cleare, S., Mitchell, G., 2011. A feasibility study of Acceptance and Commitment Therapy for emotional dysfunction following psychosis. Behav. Res. Ther. 49 (12), 901-907.

Yusupoff, L., Tarrier, N., 1996. Coping strategy enhancement for persistent hallucinations and delusions. Cognitive-Behavioural Interventions with Psychotic Disorders 86-102.

Schizophrenia Bulletin Advance Access published March 8, 2014

Feasibility, Acceptability, and Preliminary Efficacy of a Smartphone Intervention for Schizophrenia

Dror Ben-Zeev*,1, Christopher J. Brenner2, Mark Begale3, Jennifer Duffecy3, David C. Mohr3, and Kim T. Mueser1,4

'Department of Psychiatry, Dartmouth Psychiatric Research Center, Geisel School of Medicine at Dartmouth, Lebanon, NH; 2Thresholds, Chicago, IL; 3Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL; 4Center for Psychiatric Rehabilitation, Sargent College of Health and Rehabilitation Sciences, Boston University, Boston, MA

*To whom correspondence should be addressed; Dartmouth Psychiatric Research Center, 85 Mechanic Road, Lebanon, NH 03766, US; tel: 603-448-0263, fax: 603-448-3976, e-mail: dror.ben-zeev@dartmouth.edu

The FOCUS smartphone intervention was developed to provide automated real-time/real-place illness management support to individuals with schizophrenia. The system was specifically designed to be usable by people with psychotic disorders who may have cognitive impairment, psychotic symptoms, negative symptoms, and/or low reading levels. FOCUS offers users both prescheduled and on-demand resources to facilitate symptom management, mood regulation, medication adherence, social functioning, and improved sleep. In this study, 33 individuals with schizophrenia or schizoaffective disorder used FOCUS over a 1month period in their own environments. Participants were able to learn how to use the intervention independently, and all but one participant completed the trial successfully and returned the smartphones intact. Completers used the system on 86.5% of days they had the device, an average of 5.2 times a day. Approximately 62% of use of the FOCUS intervention was initiated by the participants, and 38% of use was in response to automated prompts. Baseline levels of cognitive functioning, negative symptoms, persecutory ideation, and reading level were not related to participants’ use of the intervention. Approximately 90% of participants rated the intervention as highly acceptable and usable. Paired samples t tests found significant reductions in psychotic symptoms, depression, and general psychopathology, after 1 month of FOCUS use. This study demonstrated the feasibility, acceptability, and preliminary efficacy of the FOCUS intervention for schizophrenia and introduces a new treatment model which has promise for extending the reach of evidence-based care beyond the confines of a physical clinic using widely available technologies.

Key words: Mobile Health (mHealth)/mobile interventions/depression/hallucinations/medication adherence/sleep/social functioning

Introduction

Schizophrenia is associated with high costs to individuals, their families, and society.1,2 Poorly managed, the illness can cause significant personal distress and impairment and is associated with increased risk for depression, anxiety, substance use, homelessness, victimization, hospitalization, and suicide.3-8

Over the last 2 decades several evidence-based psychosocial interventions have been developed to help individuals with schizophrenia better cope with symptoms, improve social functioning, maintain a healthier lifestyle, and engage in meaningful work, even in the context of a chronic mental health condition.9,10 However, these interventions are rarely available at clinical settings for a variety of reasons, including the lack of clinicians who are trained in these approaches, limited funding for psychosocial interventions, and poor utilization and ongoing engagement in treatments even when they are available.11-13

Mobile Health (mHealth) approaches that leverage mobile devices such as cellular phones and smartphones to support healthcare are promising for deployment of interventions that are unconstrained by the limitations of existing treatment settings.14,15 Mobile phones are carried on the person, typically turned on, and have near constant connectivity and access to multimedia resources. Thus, they can serve as conduits for interventions any time, and in almost any location.16 Furthermore, mobile phones are widely available, affordable, and are continuously dropping in cost; there are now over 6 billion mobile phones subscriptions worldwide, with the majority being used in low and middle income countries.17 In the United States, underserved populations now use “smartphones” (ie, mobile phones with computational capacities) as their primary method for accessing resources on the internet,18 and there is evidence that even homeless individuals with limited resources own and use mobile phones regularly.19-20 Mobile phone use among people with severe psychiatric disabilities is not dramatically different from that observed in the general population—a recent survey conducted among 1592 adults with serious mental illnesses found that 72% of respondents had mobile phones and used them for a range of functions including calling, texting, and email. Moreover, many individuals indicated that they would be interested in engaging in mobile interventions (eg, reminders, psychoeducation, contact with clinicians) via their mobile device.21

© The Author 2014. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com


Early efforts to incorporate mobile phones into the clinical care of people with schizophrenia have produced mixed results. In one pilot study, participants received daily text message assessments sent to basic mobile phones from a remote server.22 If participants responded, the server engaged them with follow-up suggestions to improve their symptoms and functioning. Participants responded to 86% of assessments over 3 months, but clinical outcomes did not significantly change from baseline to posttrial. Some changes in self-reported distress from auditory hallucinations, social functioning, and medication adherence were recorded, but participants with lower cognitive functioning and more severe negative symptoms had greater difficulty negotiating the requirements of the protocol. The investigative team concluded that there was a need to develop interventions that capitalize on emerging smartphone technology that could enable more userfriendly and potent interventions.

In a second study, high risk for relapse outpatients and their family members received weekly text message requests to complete assessments of early warning signs of relapse on their mobile phones as part of a randomized controlled trial of an information technology-aided relapse prevention program.23 When participants reported problems, automated alerts were sent to their treating psychiatrists with a prompt to follow-up with a medication evaluation. Participants responded to 80% of weekly requests over a year, but practitioner adherence to the protocol was low and in many cases follow-up steps were not taken. When clinicians did respond accordingly, individuals in the intervention arm did significantly better than controls in terms of hospitalization, number of inpatient days, and costs. Taken together, these studies demonstrate that people with schizophrenia can successfully engage in mobile interventions, but that these should be user-centered in terms of the technology (ie, user-friendly, intuitive, engaging) and treatment model (ie, include self-management components that are independent of clinician engagement).

FOCUS: A User-Centered Smartphone System for Schizophrenia

Working together with patients and clinicians at community settings, our research team developed FOCUS, a smartphone system designed to support self-management of illness in individuals with schizophrenia. The system is grounded in 2 theoretical models: the cognitive model of psychosis24,25 and the stress-vulnerability model of schizophrenia.26,27 FOCUS aims to identify and dismantle dysfunctional beliefs that contribute to maintenance and distress associated with symptoms, and to interrupt the cyclical relationship between stress (eg, fatigue, interpersonal conflict, social isolation, poor medication adherence) and vulnerability that may lead to illness exacerbation.28 Through several iterative cycles of development and user feedback, we constructed a smartphone system that targets symptoms of psychosis, social functioning, mood problems, medication adherence, and sleep difficulties. We drew treatment content from an array of evidence-based interventions (ie, cognitive restructuring, behavioral tailoring, social skills training, illness management and recovery, anger management, behavioral activation, sleep hygiene), and adapted it so that it was suitable for delivery via smartphones. Content was distilled into brief interactive exchanges that are accompanied by illustrative images (ie, photographs, cartoons, touchscreen reminder buttons) that are displayed on the smartphone screen. To maximize usability, the system was developed in accordance with design principles for electronic resources for people with serious mental illness and cognitive impairment.29

FOCUS was initially tested with individuals with schizophrenia in laboratory conditions, and the system was adapted based on our observations of problems and user recommendations.30 Once we were satisfied that the system functioned well in controlled environments, we conducted a field trial in which 33 participants were provided with a smartphone installed with the FOCUS system, to use over 1 month in their own environments. We hypothesized that participants would find the system acceptable, usable, engaging, and helpful. We also hypothesized that participants’ cognitive functioning, negative symptoms, and levels of persecutory ideation would not impact their use of the system. To our knowledge, this is the first deployment of a smartphone intervention for schizophrenia.

Methods

Participants

The study was approved by the Committee for Protection of Human Subjects at Dartmouth. Thirty-three individuals with schizophrenia or schizoaffective disorder were recruited from community-based treatment programs in Chicago. Participants had a mean age of 45.9 (SD = 8.78). The sample was 61% male, 76% African American, 21% white, and 3% more than one race. Six percent identified as Latino. Participants reported an average of 12.7 years of education (SD = 2.32), but their average reading level was at the eighth grade level (assessed using the reading subsection in Wide Range Achievement Test—Fourth Edition, Wilkinson and Robertson31 [WRAT-4]). Sixty-one percent were living independently, 21% resided in a supervised living facility, and 18% were living with family members. The majority were unemployed (87.9%) and owned a mobile phone of some kind (87.5%). Of mobile phone users, 32% owned a smartphone. Participants reported an average of 6.40 (SD = 4.21) lifetime psychiatric hospitalizations. At baseline, the sample experienced moderate symptoms of schizophrenia (Positive and Negative Syndrome Scale, Kay et al32 [PANSS] total, M = 77.64, SD = 4.23; PANSS-positive score, M = 19.24, SD = 4.23; PANSS-negative score, M = 17.76, SD = 3.60), mild depressive symptoms (Beck Depression Inventory-Second Edition, Beck et al33 [BDI-2], M = 19.52, SD = 8.86), and subthreshold clinical insomnia (Insomnia Severity Index, Morin et al34 [ISI], M = 12.27, SD = 6.24). Participants’ beliefs that it was necessary for them to take medications were stronger than their medication-related concerns (Brief Medication Questionnaire, Svarstad et al35 [BMQ] necessity-concern differential, M = 5.15, SD = 5.69). On average, participants had moderate cognitive impairment (Brief Assessment of Cognition in Schizophrenia36 [BACS] t-score, M = 30.09, SD = 11.46).

Procedures

Participant Screening. Clinical staff identified 95 individuals who were viable candidates for the study (ie, chart diagnosis, willingness to being contacted for research projects). Research staff contacted this group, and 58 expressed interest in participating in the study. After providing written informed consent, all potential participants were administered a structured diagnostic interview (Structured Clinical Interview for DSM-IV Axis I Disorders-Patient Edition, First et al37) to verify diagnosis of schizophrenia or schizoaffective disorders. Potential participants then completed a battery of laboratory-based self-report and interview measures that included demographic information, measures of symptoms of schizophrenia (PANSS), symptoms of depression (BDI-2), and sleep difficulties (ISI). Potential participants were enrolled if they were 18 years of age or older, prescribed psychotropic medications, and had “mild” or higher severity scores on 2 of the following: hallucinatory behavior on the PANSS (P3 > 3), passive/apathetic social avoidance or active social avoidance on the PANSS (N4 or G16 > 3), BDI-2 total score > 10, or ISI score > 15. Candidates were excluded if they had hearing, vision, or motor impairments that made it impossible for them to use a smartphone (assessed on-site by study staff), if their reading level was below fourth grade, or if they were enrolled in another intervention study. Five individuals were found to be ineligible due to diagnosis, 6 were ineligible due to reading level, 13 were ineligible due to symptom severity criteria, and 1 was excluded due to enrollment in another study.

Pretrial. Once enrolled, participants returned within a week and were administered the BACS and BMQ. This visit also included a shared decision-making session with research staff to establish the treatment targets each participant would work on (ie, receive daily prompts and content from the FOCUS system) over the course of the field trial. Every participant was assigned medication adherence as a treatment target because we wanted to have at least one element that was consistent across all participants, and this was identified as a high priority area by clinical staff. Two additional domains were chosen from the following options: social (encompasses interventions targeting persecutory ideation, anger management, and social skills training), mood problems (ie, depression and anxiety), auditory hallucinations, or sleep difficulties. The domain selection was informed by participants’ preference and data from the screening and baseline assessments; a study interviewer would identify measure scores that were particularly high and record concrete examples participants provided when asked about areas of greatest concern (eg, avoiding locations where voices were loudest, panic attacks, staying up at night ruminating). The interviewer would then reflect back to the participant their assessment of 2 “high priority” areas, and inquire whether they agree and would like to focus on these targets or if they preferred a different combination. Once they decided on the domains, the interviewer and participant would review their “typical day” and discuss when it would make most sense to address each domain (ie, the times they would get daily FOCUS prompts to engage).

Following treatment target selection, a 30-min training session was administered focusing on use of the smartphone provided and the different elements of the FOCUS system. Training on the use of the device included how to charge the phone, turning the phone on and off as well as locking the screen, how to use a touchscreen, how to make and receive calls, and how to save contact numbers. The FOCUS system was then demonstrated by a trained research assistant. The demonstration focused on in-the-moment use and selection of resources from the different on-demand options. Participants then had the opportunity to practice using FOCUS and ask questions as needed. Individuals were given a Motorola Droid 4 smartphone running the Android 4.1 operating system and charger and commenced their participation in the field trial only after they demonstrated proficiency in using the device and the FOCUS system in the laboratory. They were instructed to charge the phone at night and carry it with them wherever they go, but otherwise to go about their daily life as usual.

Field Trial. Over the course of 1 month, the FOCUS system prompted participants to complete an assessment 3 times daily, on one of each of the 3 treatment targets between the hours of 9 am and 1 pm, 1 and 5 pm, and 5 and 9 pm (exact times within those ranges were determined randomly daily by the system). Participants were asked to respond to each system-generated assessment whenever possible. In response to the content of participant entries, the FOCUS system deployed tailored in-the-moment interventions. Participants were also instructed to use the host of on-demand FOCUS features as often as they liked, whenever they needed support. The FOCUS system automatically uploaded participant use data (ie, response to prompt rate, content of their responses to assessment, self-initiated use of resources) to a secure study server. Thus, so long as the smartphone was within reception, the research team could view user response data continuously. Study staff called each participant once per week to check-in and assist with any technical difficulties.

Posttrial. After 1 month of use, participants returned the charger and smartphone and completed the PANSS, BDI-2, ISI, and BMQ, for a second time. Before debriefing, participants also completed a 26-item self-report acceptability/usability measure comprised of adapted items from the System Usability Scale,38 Post Study System Usability Questionnaire,39 Technology Assessment Model Measurement Scales,40 and Usefulness, Satisfaction, and Ease questionnaire.41 Participants were asked to rate their agreement with a series of statements about the intervention (see table 1 for all items).

Participants were paid for the time they devoted to pre-and posttrial assessments. They received an unlimited data plan that enabled unrestricted calling, texting, and internet use on the smartphone during the trial. Daily engagement in the FOCUS intervention was not incentivized and participants were instructed to use the system as often as they wanted.

Description of the Mobile Intervention

The FOCUS system is comprised of 3 applications (apps) that are installed onto the smartphone, and a web-based dashboard. The first app prompts users to engage daily via auditory signals and visual notifications that appear on the screen. The second is the primary FOCUS app that uses interactive algorithms to generate brief assessments and interventions that the user progresses through using touchscreen buttons on the smartphone homescreen. The third is a Quick Tips app that allows users to access illness self-management resources and suggested coping strategies from a menu of options.

Table 1. Participant Acceptability/Usability Ratings

Statement

Number of Participants Selecting Each Response

Disagree

Neutral

Agree

I think that I would like to use FOCUS often

1 (3.1%)

7 (21.9%)

24 (75%)

I found FOCUS to be very complicated

26 (81.3%)

2 (6.3%)

4 (12.4%)

I thought FOCUS was easy to use

3 (9.4%)

1 (3.1%)

28 (87.5%)

I think that I would need the support of a technical person to be able

24 (75%)

2 (6.3%)

6 (18.7%)

to use FOCUS

I found that the different parts of FOCUS work well together

0

2 (6.3)

30 (93.7)

I thought there was too much inconsistency in FOCUS

24 (75%)

3 (9.4)

5 (15.6)

I would imagine that most people would learn to use FOCUS very

0

1 (3.1%)

31 (96.9%)

quickly

I found FOCUS very awkward to use

24 (75%)

2 (6.3%)

6 (18.7%)

I felt very confident using FOCUS

2 (6.5%)

1 (3.2%)

28 (90.3%)

I needed to learn a lot of things before I could get going with FOCUS

24 (75%)

2 (6.3%)

6 (18.7%)

Overall, I am satisfied with how easy it is to use FOCUS

0

2 (6.3%)

30 (93.7%)

I was able to complete the “modules” quickly in FOCUS

1 (3.1%)

5 (15.6%)

26 (81.3%)

I felt comfortable using FOCUS

1 (3.1%)

2 (6.3%)

29 (90.6%)

It was easy to learn to use FOCUS

2 (6.9%)

2 (6.9%)

25 (86.2%)

Whenever I made a mistake using FOCUS, I could recover easily and

3 (9.4%)

6 (18.8%)

23 (81.8%)

quickly

It was easy to find the information I needed

1 (3.1%)

3 (9.4%)

28 (87.5%)

The information provided for FOCUS was easy to understand

1 (3.1%)

4 (12.5%)

27 (84.4%)

How things appeared on the screen was clear

0

1 (3.1%)

31 (96.9%)

If I have access to FOCUS, I will use it

2 (6.3%)

4 (12.5%)

26 (81.3%)

I am satisfied with FOCUS

1 (3.1%)

2 (6.3%)

29 (90.6)

I would recommend FOCUS to a friend

0

4 (12.5%)

28 (87.5)

FOCUS is fun to use

1 (3.1%)

4 (12.5%)

27 (84.4)

FOCUS works the way I want it to work

2 (6.3%)

7 (21.9%)

23 (81.8)

I feel I need to have FOCUS

8 (25%)

4 (12.5%)

20 (62.5)

FOCUS helped me manage my symptoms

0

4 (12.5%)

28 (87.5)

FOCUS was interactive enough

3 (9.4%)

2 (6.3%)

27 (84.3)


Once signaled by the prompting app (see figure 1), users can decide to engage or ignore the prompt. If they engage, the system will launch a brief assessment of their current status (eg, “How has your mood been today?”) with multiple choice touchscreen response options that appear below the question on the same screen. If the user endorses difficulties (eg, “Very bad. I’m very upset”) the system provides feedback (eg, “Looks like you could use some support. FOCUS is happy to help.”), followed by a more in-depth assessment (“Have you had any of these thoughts lately?”). Users’ responses determine the nature of the subsequent interventions they will receive (eg, see figure 2). Once they complete a sequence of screens, making selections as they progress and receiving interventions, the system signs off (“Thank you for using FOCUS. Have a nice day.”) until the next scheduled prompt.

Users can access all intervention content “on-demand” whenever and wherever they choose, by going to the FOCUS homescreen and selecting any of the 5 treatment target icons (see figure 1) or by accessing Quick Tips for briefer noninteractive content. Each intervention sequence has multiple wording and image variations so that users do not encounter the exact same intervention every time, even if they make similar selections (see examples for FOCUS intervention screens for each target area in figure 2). When the smartphone has wireless connectivity, the FOCUS application transmits the use data to a secure study server. The data is then displayed as a continuously updated report on a secure web-based dashboard that the research team or authorized clinicians can access at any time.

Overview of Analyses

Descriptive statistics were derived from participants’ smartphone use data to characterize feasibility and acceptability. Pearson product-moment correlation coefficients were used to examine the association between baseline cognitive functioning (BACS), negative symptoms (PANSS-negative scale), persecutory ideation (suspiciousness item from the PANSS), and FOCUS system use (ie, days used, number of times used per day). Spearman’s rank correlation coefficients were used to examine the association between reading level (WRAT-4 grade estimate) and FOCUS system use. Paired samples tests were used to test for differences between pretrial and posttrial clinical outcomes (PANSS, BDI-2, BMQ, ISI).

Results

Feasibility

One participant dropped out of the study after losing 2 study smartphones in the first week. The remaining 32 participants used the system successfully and returned the smartphone intact at the end of the trial. One participant did not use the smartphone for anything other than the FOCUS intervention. All other participants used a range of smartphone functions during the month,


Medication Use          Mood              Sleep


including calling (96.9%), texting (34.4%), email (15.2%), and accessing the internet (62.5%). System use data for 2 participants were lost due to technical problems during the automated data transfer to the study server.

Therefore, we report on FOCUS system use data for 30 individuals.

On average, these participants used the FOCUS system on 86.5% of the days they had the smartphone (week 1 average: 6.7 days, week 4 average: 5.9 days). On days FOCUS was used, participants interacted with the system an average of 5.19 times (week 1 average: 6.4 times daily, week 4 average: 4.9 times daily). Participants initiated their interactions with FOCUS on 62.5% of the times it was used (ie, using on-demand interventions) and 37.5% of use was in response to prescheduled system prompts.

Baseline cognitive functioning (ie, BACS score), negative symptoms (PANSS-negative scale score), persecutory ideation (suspiciousness item from the PANSS), and reading level (WRAT-4 grade estimate) were not significantly associated with the percentage of days participants used FOCUS, or the number of times they used the system on those days (all Ps > .05). Overall, these results suggest the FOCUS smartphone intervention is feasible among people with schizophrenia.

Acceptability and Usability

Participant responses to the acceptability/usability measure are reported in table 1. Over 90% of participants thought the different components of the intervention worked well together, that content appeared on the screen clearly, and that people could learn to use FOCUS very quickly. They reported feeling very confident, comfortable, and satisfied using the intervention. Over 87% reported that it was easy to find the information they needed, that the intervention helped them manage their symptoms, and that they would recommend the system to a friend. A minority of participants reported some difficulties: approximately 12% found the intervention to be complicated, 18% thought they needed to learn more things before they could get started and found it awkward to use, and 6% thought they needed more technical support. Overall, these results suggest that the majority of participants found the FOCUS intervention acceptable and usable.

Preliminary Efficacy

Paired samples t tests indicated significant reductions in symptoms from pretrial to posttrial on the PANSS total (P = .002), PANSS positive (P < .001), PANSS general psychopathology (P < .001), and in depression on the BDI-2 (P = .003). Scores on the PANSS-negative subscale did not significantly change. There were also no significant changes in beliefs about medications (BMQ general and BMQ necessity-concern differential scores) or in sleep difficulties (ISI) (see table 2).

To examine whether there was an association between symptom change and the frequency with which participants used the intervention, we conducted Pearson correlations between changes in BDI-2 and PANSS scores, and the percentage of days participants used the system.

Table 2. Pre- and Posttrial Clinical Measures (N = 32)

Measure

Pretrial, mean (SD)

Posttrial, mean (SD)

Positive and Negative Syndrome Scale

Positive scale

19.34 (4.26)**

16.41 (4.06)**

Negative scale

17.69 (3.64)

18.22 (3.29)

General psychopathology

40.88 (5.89)*

36.75 (5.38)*

scale

Total score

77.59 (10.44)*

71.47 (9.89)*

Beliefs About Medicines Questionnaire

Necessity-concern

5.16 (5.78)

3.53 (5.75)

differential

General total

21.28 (4.92)

21.44 (4.58)

Insomnia Severity Index

12.25 (6.34)

11.56 (8.13)

Beck Depression Inventory-2

19.69 (8.94)*

14.78 (10.28)*

*P < .01, **P < .001 (paired samples t tests).

We found a significant association between the change in participants’ BDI-2 scores and the percentage of days participants used FOCUS over the 1-month period (r = -.36, P < .05); the greater the reduction in depressive symptoms, the less often participants used the system. There was no significant association between PANSS scores and the percentage of days participants used the system.

Discussion

This study demonstrated that a smartphone intervention for illness management in individuals with schizophrenia is feasible and acceptable, and suggests the system may be clinically helpful. Participants reported high levels of satisfaction and an interest in continuing to use the smartphone system in the future. Acceptability of the smartphone intervention was high; participants used the system 86% of days they had the device. Additionally, individuals elected to use on-demand resources above and beyond the preprogrammed daily prompts; participant-initiated engagement accounted for over 60% of all intervention use. This finding is particularly important, given that participants were not given incentives to use the intervention during the trial and their access to other smartphone resources (ie, calling, texting, internet) was not dependent on their use of the FOCUS system. Participants’ use of the system was not hampered by their level of cognitive functioning, negative symptoms, persecutory ideation, or reading level.

Preliminary assessment of clinical efficacy suggested that the FOCUS intervention was helpful in reduction of positive symptoms of schizophrenia (PANSS-positive scale), general symptoms of psychopathology (PANSS general psychopathology scale), and depression (BDI-2) over the 1-month trial. There were no significant changes in sleep or beliefs about medications. Unlike the symptom-focused content, the smartphone system’s sleep interventions were designed to promote long-term lifestyle changes (ie, sleep hygiene) rather than in-the-moment coping strategies and might not provide immediate relief when deployed. For example, suggesting that an individual avoid naps during the day in order to facilitate a regular sleep/wake cycle may be less helpful in the moment when the person is struggling to fall asleep. It is also possible that positive sleep outcomes take longer than a month to emerge, and a longer trial would have produced stronger effects. At the beginning of the study most participants had stronger beliefs that medications were necessary for them to stay healthy than they had concerns about their use (BMQ necessity-concern differential), making it difficult to detect further increases in the importance of medication.

Overall, the clinical outcomes reported in this study are comparable or better than those produced by other psychosocial interventions9,10 and required a fraction of the cost. Even after accounting for the price of smartphone devices (and their possible replacement), data plans, and technical support staff needed to deploy FOCUS, it is much less resource intensive than services provided by a mental health professional at a clinic setting. Moreover, unlike scheduled face-to-face services, a mobile intervention is transportable and can be used in any location.

While the FOCUS system was used in the study as an adjunct to in-person services, one could also envision a future where evidence-based mHealth apps such as FOCUS are downloaded directly onto smartphones and used by individuals with little or no access to any mental health care. As familiarity with mobile technology increases, so may the range of potentially therapeutic options for using them to promote coping with psychiatric illnesses. For people who are now growing up with mobile technologies in hand (ie, “digital natives”) using the full range of smartphone capabilities will be intuitive (eg, uploading photos, using tools to adapt or generate their own content, connecting to social media). As a result, the level of sophistication and potential impact of mHealth interventions (ie, tailored, personalized, adaptive) will increase.

This study has several limitations. First, there was no control group, so it is not possible to determine whether the clinical improvements were related to use of the FOCUS system. Future research will need to evaluate FOCUS in more rigorously controlled studies and to examine whether symptom improvements persist over time. Second, the system was deployed for a relatively short amount of time. While it is encouraging to see some rapid therapeutic gains, future research will need to examine whether people find smartphone mHealth systems engaging and helpful over extended periods, or whether these are most suitable for time-limited care (eg, upon discharge from inpatient care, during symptom exacerbations). Third, the sample size was relatively small, impacting generalizability. It is also possible that a larger sample would be sufficiently powered to detect small changes in clinical outcomes that were not significant in the current study (eg, sleep ratings). However, whether these small improvements are clinically meaningful, is questionable. Finally, the clinical rater was not blinded to the study objectives and the nature of the intervention. This might have impacted their ratings on posttrial measures that require some clinical interpretation (ie, PANSS).

Clinical researchers have begun developing a number of other novel technologies to improve the accessibility and quality of care available to people with schizophrenia.42,43 Several of these approaches include web-based self-paced cognitive behavioral interventions for auditory hallucina-tions,44 online peer support and social therapy for first-episode psychosis,45 internet-based family intervention programs,46,47 computerized “relational agents” designed to enhance medication adherence and physical activity,48 and virtual reality paradigms for vocational rehabilita-tion49,50 and treatment of persecutory ideation.51 Many of these approaches can be adapted to smartphone and other mobile platforms that would allow patients to use them wherever and whenever they need them the most.

Conclusion

To our knowledge, this is the first study to demonstrate the feasibility, acceptability, and efficacy of a smartphone intervention for schizophrenia. Incorporating a user-centered approach in the intervention development process was essential to generating a system that can address many of the unique obstacles that people with schizophrenia face when attempting to engage in mHealth treatment. The integration of several adapted evidence-based psychosocial intervention strategies into a single mobile platform appeared promising. But the FOCUS system does not merely serve as a delivery system for existing interventions. Rather, it introduces a novel approach to clinical care for schizophrenia (ie, real-time, real-place, on-demand, self-navigated), ie, only made possible through recent advancements in mobile hardware, software, and telecommunication infrastructure. As smartphone and other mobile technologies continue to develop, they will create new and exciting opportunities for innovative mHealth systems that will enable continuous assessment and treatment. Leveraging both active (ie, self-report of symptoms and functioning) and passive (ie, sensors that capture behavior and physiology) patient data will undoubtedly increase the potency of treatments for people with schizophrenia in the years ahead.

Funding

National Institute of Mental Health (R34MH100195, Principal Investigator: D.B.-Z.); Dartmouth SYNERGY Center for Clinical and Translational Science.

Acknowledgment

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

comorbidities and schizophrenia. Schizophr   Bull.

2009;35:383-402.

. Alvarez-Jimenez M, Bendall S, Lederman R, et al. On the HORYZON: moderated online social therapy for longterm recovery in first episode psychosis. Schizophr Res. 2013;143:143-149.

. Glynn SM, Randolph ET, Garrick T, Lui A. A proof of concept trial of an online psychoeducational program for relatives of both veterans and civilians living with schizophrenia. Psychiatr Rehabil J. 2010;33:278-287.

. Rotondi AJ, Anderson CM, Haas GL, et al. Web-based psy-choeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv. 2010;61:1099-1105.

. Bickmore TW, Puskar K, Schlenk EA, Pfeifer LM, Sereika SM. Maintaining reality: relational agents for antipsychotic medication adherence. Interact Comput. 2010;22:276-288.

. Bell MD, Weinstein A. Simulated job interview skill training for people with psychiatric disability: feasibility and tolerability of virtual reality training. Schizophr Bull. 2011;37(suppl

. Tsang MMY, Man DWK. A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophr Res. 2013;144:51-62.

. Freeman D. Studying and treating schizophrenia using virtual reality: a new paradigm. Schizophr Bull. 2008;34:605-610.

Original Paper

Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood and Psychotic Disorders

Dror Ben-Zeev1, PhD; Benjamin Buck1,2,3, PhD; Phuonguyen Vu Chu1, BA; Lisa Razzano4,5, CPRP, PhD; Nicole Pashka5, MS, CRC, CPRP, LCPC; Kevin A Hallgren1, PhD

1Behavioral Research In Technology and Engineering Center, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States

2Health Services Research and Development, VA Puget Sound Healthcare System, Seattle, WA, United States 3Department of Health Services, School of Public Health, University of Washington, Seattle, WA, United States 4Center on Mental Health Services Research and Policy, Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, United States 5Thresholds Inc, Chicago, IL, United States

Corresponding Author:

Dror Ben-Zeev, PhD

Behavioral Research In Technology and Engineering Center

Department of Psychiatry and Behavioral Sciences

University of Washington

1959 NE Pacific Street

Seattle, WA, 98195

United States

Phone: 1 206 685 9655

Email: dbenzeev@uw.edu

Abstract

Background: Depression is the most prevalent mental health problem. The need for effective treatments for depression far outstrips the availability of trained mental health professionals. Smartphones and other widely available technologies are increasingly being leveraged to deliver treatments for depression. Whether there are patient characteristics that affect the potency of smartphone interventions for depression is not well understood.

Objective: This study aimed to evaluate whether patient characteristics including clinical diagnosis, depression severity, psychosis status, and current use of antidepressant medications impact the effects of an evidence-based smartphone intervention on depressive symptoms.

Methods: Data were collected as part of a 2-arm randomized controlled trial comparing a multimodal smartphone intervention called FOCUS with a clinic-based intervention. Here, we report on 82 participants assigned to 12 weeks of FOCUS treatment. We conducted assessments of depressive symptoms using the Beck Depression Inventory-second edition (BDI-II) at baseline, postintervention (3 months), and follow-up (6 months). We tested for differences in the amount of improvement in BDI-II scores from baseline to posttreatment and 6-month follow-up between each of the following patient subgroups using 2 (group) x 2 (time) mixed effects models: diagnosis (ie, schizophrenia spectrum disorder vs bipolar disorder vs major depressive disorder), depression severity (ie, minimal-mild vs moderate-severe depression), psychosis status (ie, presence vs absence of psychotic symptoms), and antidepressant use (ie, taking antidepressants vs not taking antidepressants).

Results: The majority of participants were male (60%, 49/82), African American (65%, 53/82), and middle-aged (mean age 49 years), with a high school education or lower (62%, 51/82). There were no differences in patient demographics across the variables that were used to stratify the analyses. There was a significant group x time interaction for baseline depression severity (Fb768=5.26, P=.02 [posttreatment] and Fj 774=6.56, P=.01 [6-month follow-up]). Participants with moderate or severe depression had significant improvements (t42=3.20, P=.003 [posttreatment] and /42=4.20, P<.001 [6-month follow-up]), but participants with minimal or mild depression did not (t31=0.20, P=.84 [posttreatment] and /30=0.43, P=.67 [6-month follow-up]). There were no significant group x time interactions for diagnosis, psychosis status, or antidepressant medication use. Participants with minimal or mild

http://mental.jmir.org/2019/4/e13202/

XSL-FO

depression had negligible nonsignificant improvements (<1 point on the BDI-II). Reduction in depression in all other groups was larger (range 1.7-6.5 points on the BDI-II).

Conclusions: Our results suggest that FOCUS can be deployed to treat moderate to severe depressive symptoms among people with schizophrenia spectrum disorders, bipolar disorder, and major depressive disorder, in concert with antidepressant medications or without them, in both people with and without active psychotic symptoms. The study results are consistent with research on transdiagnostic models in psychotherapy and extend our knowledge about the potential of transdiagnostic mobile health.

Trial Registration: ClinicalTrials.gov NCT02421965; http://clinicaltrials.gov/ct2/show/NCT02421965 (Archived by WebCite at http://www.webcitation.org/76pyDlvAS)

(JMIR Ment Health 2019;6(4):e13202) doi:10.2196/13202

KEYWORDS

mHealth; schizophrenia; bipolar disorder; depression; illness management; symptoms; transdiagnostic

Introduction

Background

Depression is a universal experience. Symptoms of depression are the most common mental health concern reported across nationalities, ethnicities, and age groups worldwide [1,2]. Depression is prevalent in all clinical settings, either as the primary issue that brings people to seek mental health care [3,4] or as a significant comorbid concern that emerges in those contending with medical illnesses [5-8], physical disabilities [9,10], relationship problems [11], work and educational difficulties [12], or substance use problems [13]. Depression is a source of enormous financial and societal burden. It is the second leading cause of years lived with disability worldwide [14], with overall medical and job loss costs estimated at US $210.5 billion annually [15]. It is also implicated in half to two-thirds of all completed suicides [16]. The need for effective treatments for depression far outstrips the availability of resources that can be delivered by trained mental health professionals. This tension has led to the emergence of new models of care that are no longer dependent on the availability of highly trained mental health specialists, including integration of depression treatments into the offerings of primary care settings [17-19], training of paraprofessionals and family members to provide support [20], and using new technology to expand the breadth and reach of depression management resources [21,22].

It is only fitting that in recent years the most common mental health problem is starting to be addressed with the aid of one of the most widely used technologies on the planet: the mobile phone [23,24]. Smartphones—contemporary mobile phones with multimedia players, internet connectivity, and the ability to host apps—are increasingly being leveraged to deliver treatments for depression. A recent meta-analysis of 18 randomized controlled trials (RCTs) of smartphone-based mental health interventions for depressive symptoms found that these treatments had positive effects in comparison with both active interventions and inactive control conditions. Therapeutic effects were found for those with self-reported mild to moderate depression but were not seen among those with diagnoses of major mood disorders [25]. The authors of the meta-analysis outlined that these findings may be linked to the small and underpowered subsample sizes used in the studies involving those with clinical diagnoses and emphasized the need for further research to deepen our understanding of which populations stand to benefit the most from smartphone interventions for depressive symptoms. Digital treatments are very novel in mental health care, and there is still uncertainty about whether these approaches are appropriate for all patients [26,27]. Research evaluating the effects of mobile health (mHealth) on people with both mild and more severe symptomatology can address this gap in our knowledge.

Objectives

Given that depressive symptoms commonly emerge in many forms of psychopathology [28], in this study, we examined the effects of an evidence-based smartphone intervention called FOCUS on depressive symptoms transdiagnostically among people with mood and psychotic disorders. We evaluated whether several patient clinical characteristics (ie, diagnosis, depression symptom severity, psychosis symptoms, and antidepressant medication use) impact the effects of the smartphone intervention on depressive symptoms in an RCT.

Methods

Study Descriptions

We conducted an assessor-blind, 2-arm, RCT between 2015 and 2017. The project was conducted in partnership with a large Chicago-based mental health agency that provides services to a range of people with psychiatric conditions. The study was approved by the Institutional Review Boards of the University of Washington and Dartmouth College and monitored continuously by an independent safety monitoring board. All study participants completed informed consent. Individuals were randomized (1:1 ratio) into 1 of the 2 treatment arms: an mHealth intervention delivered via smartphone (FOCUS) or a clinic-based group intervention (Wellness Recovery Action Plan). Interventions were deployed for a period of 12 weeks. We conducted assessments at baseline, postintervention (3 months), and follow-up (6 months). Participants were compensated $30 per assessment. The study was registered in ClinicalTrials.gov, and the main RCT comparison outcomes were reported in an earlier publication [29]. Here, we conducted a secondary analysis focusing specifically on patients who received the FOCUS smartphone intervention, examining whether several patient clinical characteristics preintervention affected the impact of the intervention on their depressive symptoms.

Smartphone Intervention

FOCUS is a multimodal, smartphone-delivered intervention that was originally designed to support the recovery of people with schizophrenia [30] but has since been deployed among multiple diagnostic groups [29,31,32]. The FOCUS intervention comprises a FOCUS app that is used independently by patients, a Web-based clinician dashboard that summarizes participants’ responses to self-assessments and their use of various FOCUS resources, and an mHealth support specialist who helps participants make meaningful use of the FOCUS intervention and provides technical troubleshooting assistance via brief weekly phone calls [32]. FOCUS treatment content targets 5 broad domains: mood (ie, depression and anxiety), voices (ie, auditory hallucinations), sleep problems, medication use, and social functioning. Content takes the form of either brief video, audio, or sequences of digital screens with written material coupled with visual displays. The FOCUS app includes preprogrammed daily prompts (questions that take over the home screen), followed by tailored intervention content. Participants who identify as having significant difficulties with depression at baseline may be assigned mood focused prompts, but all content is accessible on demand to all users 24/7 without restriction.

Participants

A total of 82 participants who were assigned to the FOCUS smartphone arm in the RCT are included in this report; posttreatment and 6-month follow-up data were available for 75 (91.5%) and 74 (90.2%) participants, respectively. Participants were identified by research staff and clinical teams at 3 agency sites. Study inclusion criteria included the following: chart diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, or major depressive disorder; aged 18 years or older; and a rating of “3” or lower on 1 of the 3 items comprising the Domination by Symptoms factor from the Recovery Assessment Scale [33]. Exclusion criteria included hearing, vision, or motor impairment (ie, that could affect the operation of a smartphone); less than 5th-grade English reading ability (per the Wide Range Achievement Test-4) [34]; and exposure to study interventions in the past 3 years. Participants continued to be eligible for all other clinical services including crisis intervention, assertive community treatment, supported employment, psychiatric evaluation, medication monitoring, psychosocial rehabilitation, and case management. Services were delivered in-person in the community or at 1 of the agency’s multiple locations.

Measures

The primary outcome (depression symptoms) was measured with the Beck Depression Inventory-second edition (BDI-II) [35]. The BDI-II is a self-report questionnaire with 21 items rated on a 4-point scale that can be summed for a continuous total depression severity score ranging from 0 to 63; scores can also be categorized to characterize symptom severity (0-13=minimal, 14-19=mild, 20-28=moderate, and 29-63=severe). For subgroup analyses, participants were categorized based on whether they had minimal to mild depression versus moderate to severe depression at baseline. Psychotic symptoms were assessed with the Psychotic Symptom Rating Scales (PSYRATS) [36], a semistructured interview instrument that assesses the severity of auditory hallucinations (eg, frequency, duration, loudness, and distress) and delusions (eg, preoccupation, conviction, and disruption). The PSYRATS comprises 17 items, each rated on a 4-point scale and summed for a total psychotic symptoms score. Given the distribution of psychotic symptoms at baseline in our sample (63.8% endorsing none), we dichotomized our sample based on whether individuals had any psychotic symptoms (vs none) at baseline. Antidepressant medication use was recorded by study assessors during baseline interviews and follow-up calls where participants were asked to read their medication labels to study staff. We dichotomized our sample based on whether study participants were actively taking any antidepressant medications (vs not) before commencing study interventions. Participants’ diagnosis was recorded from the electronic health records. Diagnoses are determined by licensed clinical social workers or licensed clinical professional counselors who interview clients about their mental health challenges and history, examine any prior medical records, and consult with the agency’s medical director who is a board-certified psychiatrist.

Data Analytic Plan

We conducted a series of 2 (group) x 2 (time) mixed effects models to evaluate whether there were differences in the amount of clinical improvement in BDI-II scores between each of the following groups: (1) Diagnosis, schizophrenia spectrum disorder versus bipolar disorder versus major depressive disorder; (2) Depression severity, minimal-mild versus moderate-severe depression; (3) Psychosis symptoms, presence versus absence of psychotic symptoms, and (4) Antidepressant medications, taking antidepressants versus not taking antidepressants. Any significant group x time interactions would indicate that the amount of clinical improvement was moderated by the baseline group variable; interactions that were significant were followed by paired sample t tests to evaluate the significance of the amount of changes in BDI-II scores within each specific group.

Results

Demographics and Study Variables

Descriptive statistics are presented in Table 1. The majority of participants were male and African American; the mean age was 49 years. Most participants had a high school education or lower (62%, 51/82) and had used a smartphone before entering the study (73%, 60/82). The 3 diagnostic categories specified as inclusion criteria (schizophrenia, schizoaffective disorder, bipolar disorder, and major depressive disorder) were well represented within the sample. There were no differences in patient demographics across the baseline measures that were used to stratify our analyses (diagnosis, depression severity, psychosis symptoms, and antidepressant medications).

Table 1. Descriptive statistics at baseline (N=82).

Demographic and study variables

Statistics

Age (years), mean (SD)

49 (10.1)

Male, n (%)

49 (60)

Previously used smartphone, n (%)

60(73)

Race, n (%)

White

22 (27)

African American

53(65)

Other or more than 1 race

7 (9)

Education, n (%)

High school or less

51 (62)

More than high school

31 (38)

Diagnoses, n (%)

Schizophrenia/schizoaffective disorder

38 (46)

Bipolar disorder

21 (26)

Major depressive disorder

23 (28)

Depression, n (%)

Minimal or mild

35 (43)

Moderate or severe

47(57)

Psychosis symptoms, n (%)

Absent

51 (62)

Present

31 (38)

Antidepressant medications, n (%)

None

34 (41)

One or more

46 (56)

Unknown

2 (2)

6-Month Follow-Up and Posttreatment Results

As was also reported in the parent trial [29], participants in the FOCUS condition had significant reductions in BDI-II scores at posttreatment (mean change=-2.72; t74=-2.80; P=.006) and 6 months (mean change=-4.03; t73=-3.53; P<.001) over baseline, indicating that for the full sample, depression symptoms improved during the FOCUS intervention and that these improvements were maintained at the 6-month follow-up. There were no significant group x time interactions for diagnostic group (F2738=0.16, P=.86 [posttreatment] and F2745=1.07, P=.35 [6-month follow-up]), psychosis symptoms (F1 75 1=1.89, P=.17 [posttreatment] and F1750=0.70, P=.41 [6-month follow-up]), or antidepressant medication use (Ft773<0.01, P=.95 [posttreatment] and F1800=0.20, P=.65) [6-month follow-up]), indicating that there were no significant differences in the amount of improvement participants experienced over time between the subgroups that were defined


by these 3 baseline variables. However, there was a significant group x time interaction for baseline depression severity (F1768=5.26, P=.02 [posttreatment] and Ft774=6.56, P=.01 [6-month follow-up]), indicating that the amount of improvement in depression scores was different between participants with minimal or mild depression symptoms compared with participants with moderate or severe depression symptoms. Follow-up tests within these groups indicated that participants with minimal or mild depression did not have significant reductions in depression symptoms from baseline to posttreatment (difference=-0.22; t31=-0.20; P=.84) or follow-up (difference=-0.65; t30=-0.43; P=.67); however, participants with moderate or severe depression did have significant reductions in depression symptoms at posttreatment (difference=-4.58; t42=-3.20; P=.003) that were also maintained at follow-up (difference=-6.57; t42=-4.20; P<.001). Average levels of change within all subgroups (with 95% CIs) are further characterized (see Figure 1).


http://mental.jmir.org/2019/4/e13202/

XSL-FO


Discussion

Principal Findings

The results of this study deepen our understanding of patient clinical characteristics that may impact the effectiveness of FOCUS on their level of depression, independent of their assigned diagnostic label. We found that (1) FOCUS produced significant and sustained (at 6-month follow-up) reduction in depression among people who had moderate to severe depressive symptoms, effects that were not seen among people with minimal to mild depressive symptoms; (2) FOCUS produced significant and sustained reductions in depression among people with schizophrenia/schizoaffective disorder, bipolar disorder, and major depressive disorder; (3) FOCUS produced significant and sustained reductions in depression among people with psychotic symptoms and among people without any indication of psychosis; and (4) FOCUS produced significant and sustained reduction in depression in both people who were taking antidepressant medications and people who were not.

Our findings suggest that FOCUS might be a useful intervention to address moderate to severe depressive symptoms among individuals with an array of mental illnesses. Depressive symptoms are common among people experiencing psychosis [37], are linked with poorer outcomes [38,39], and often persist or recur even with antidepressant treatment [40]. Our findings go against current skepticism about the viability of computerized interventions for people with psychosis [26] as FOCUS produced significant positive effects in both individuals with and without active psychotic symptoms and in both people with and without a schizophrenia spectrum diagnosis.

In the context of precision medicine in mental health care [41] and the growing interest in customization of treatment for well-defined populations, this study can inform practical clinical decision making. Our results suggest that FOCUS can be deployed to effectively treat depression transdiagnostically among people with moderate to severe depressive symptoms, in concert with antidepressant medications or without them, in both people with and without co-occurring psychotic symptoms.

Limitations

The study has several limitations. First, because mHealth was adjunctive to existing service provided through the community agency, other services may also have contributed to the positive changes that occurred during the study period. Second, the original study was designed with sufficient power to detect treatment changes in the overall sample, and thus, the subgroup analyses presented here should be interpreted with caution. Finally, dichotomous groups based on baseline variables were broad; a larger sample would allow examination of more fine-grained or continuous relationships between demographic or clinical characteristics and treatment benefit.

Conclusions and Future Directions

FOCUS was designed to maximize accessibility for those who are most impaired [30] while targeting several domains that are relevant transdiagnostically. Multicomponent mHealth systems are needed for users who may have diverse and evolving cognitive, emotional, and behavioral challenges. The study results are consistent with research on transdiagnostic models in clinic-based psychotherapy [42] and computerized interventions [43,44] and extend what we know about transdiagnostic mHealth. The findings also contribute to our growing awareness that mental health difficulties are multidimensional [45]. As we uncover more about heterogeneity within clinical conditions and advance our understanding of dimensionality in psychopathology, we will increasingly move away from categorical conceptualizations of “healthy” versus “ill” and “diagnosis A” versus “diagnosis B” [46]. Our mHealth interventions will likely follow suit, and like FOCUS, will continue to evolve into multidimensional and multicomponential systems [47,48]. As such, mHealth will become a more versatile mental health management approach that can serve a broader spectrum of needs.

Acknowledgments

The authors would like to thank the staff and members of Thresholds in Chicago for participating and contributing to the study.

Conflicts of Interest

DB-Z consults for eQuility and has had an intervention content licensing agreement with Pear Therapeutics. KH has provided consultation to Pear Therapeutics.

References

http://mental.jmir.org/2019/4/e13202/

XSL-FO

http://mental.jmir.org/2019Z4/e13202/

XSL-FO

Abbreviations

BDI-II: Beck Depression Inventory-second edition

mHealth: mobile health

PSYRATS: Psychotic Symptom Rating Scales

RCT: randomized controlled trial

Edited by J Torous; submitted 20.12.18; peer-reviewed by J Firth, D Fulford, K Mishina; comments to author 27.01.19; revised version received29.01.19; accepted 12.02.19; published 08.04.19

Please cite as:

Ben-Zeev D, BuckB, Chu PV, Razzano L, Pashka N, Hallgren KA

Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood andPsychotic Disorders JMIR MentHealth 2019;6(4):e13202

URL: http://mental.jmir.org/2019/4/e13202/

doi:10.2196/13202

PMID:

©Dror Ben-Zeev, Benjamin Buck, Phuonguyen Vu Chu, Lisa Razzano, Nicole Pashka, Kevin A Hallgren. Originally published in JMIR Mental Health (http://mental.jmir.org), 08.04.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.

http://mental.jmir.org/2019Z4/e13202/

XSL-FO

Original Paper

Creating Live Interactions to Mitigate Barriers (CLIMB): A Mobile Intervention to Improve Social Functioning in People With Chronic Psychotic Disorders

Bruno Biagianti13 45 6, MD; Danielle Schlosser1, PhD; Mor Nahum3,4, PhD; Josh Woolley1*, MD, PhD; Sophia Vinogradov7*, MD

department of Psychiatry, University of California, San Francisco, San Francisco, CA, United States

department of Psychiatry, University of Milan, Milano, Italy

3Posit Science, Inc., San Francisco, CA, United States

4Hebrew University, Jerusalem, Israel

department of Psychiatry, University Of Minnesota, Minneapolis, MN, United States

*

these authors contributed equally

Corresponding Author:

Bruno Biagianti, MD

Department of Psychiatry

University of California, San Francisco

401 Parnassus Avenue LP-253

San Francisco, CA, 94143

United States

Phone: 1 4152902122

Fax: 1 4152902122

Email: bruno.biagianti@ucsf.edu

Abstract

Conclusions: It is feasible and acceptable to remotely deliver an intervention aimed at enhancing social functioning in people with CPD using mobile devices. This approach may represent a scalable method to increase treatment access and adherence. Our pilot data also demonstrate within-group gains in some aspects of social cognition after 6 weeks of CLIMB. Future randomized controlled studies in larger samples should evaluate the extent to which CLIMB significantly improves social cognition, symptoms, and quality of life in CPD.

(JMIR Ment Health 2016;3(4):e52) doi:10.2196/mental.6671

KEYWORDS

psychosis; social cognition; digital health; mobile health

Introduction

Individuals with chronic psychotic disorders (CPD) struggle with poor social functioning, namely in their ability to engage in social interactions and to create meaningful relationships [1]. Contributors to poor social functioning in CPD include pervasive impairments in social cognition [2], including the perception, interpretation and processing of socially-relevant information [3]. For example, individuals with CPD show impairments in gaze perception [4], emotion perception [5,6], social cue perception [7], theory of mind [8], attribution style [9] and empathy [10]. Impairments across all of these core domains of social cognition persist throughout the course of the illness and have been linked to low occupational status, impaired community functioning, reduced capabilities for independent living, high relapse rates and reduced quality of life [1,11-14]. As a result, integrating treatments for social cognition into psychosocial interventions may be critical to improving social functioning for individuals with CPD [15].

Among the various interventions developed to improve social cognition over the past two decades (for a review, see [16]), integrated psychological therapy (IPT) [17] and cognitive enhancement therapy (CET) [18] are two models that emphasize that treatments for social cognition should take place within a meaningful social context so that patients can practice the trained social cognitive abilities in supervised real-world social situations [19]. The goal of this approach is to increase the likelihood of skill transfer to everyday life settings and to promote successful participation in real-world social situations. For example, in CET, computer-based training of social cognition is integrated with group work on social skills development. In these weekly groups, patients with CPD practice structured social interactions, solve real-life social dilemmas, do appraisal of affect and social contexts, initiate and maintain conversations and receive feedback from other patients and coaches. Integrating the group sessions has been shown to be critically important for CET to generalize to real-life accomplishments in social and occupational roles [18], although a recent study suggests that combining elements of computer-based cognitive training and social skills groups did not induce greater benefits than cognitive training alone [20]. Finally, group-based interventions are known to sustain engagement and to reduce stigma and isolation [21-23].

Research overall demonstrates positive effects of these group-based integrated interventions on social cognitive and functional outcome measures [17,18]. Unfortunately, several factors limit access to and engagement with these treatments. First, many of these interventions are currently offered in only a few specialized programs, and may not be accessible to people who live in rural or under-resourced areas [24]. Second, these programs require specially trained therapists, who may not be available in all clinical settings [25]. Third, these interventions have a high scheduling burden, usually requiring a commitment of several months (and up to two years), in-person weekly visits to clinics and the organization of patient groups for program delivery. This scheduling burden can become untenable for people who are employed, have caregiver demands, have other responsibilities to manage or are without transportation. Lastly, some individuals with CPD hesitate to approach traditional mental health treatment settings because of stigma, which interferes with help-seeking behaviors [26].

Recent advances in digital technology and mobile platforms can help overcome these limitations by supporting the delivery of interventions remotely to individuals with CPD who are unable or unwilling to come in to the clinic, and do not own or have easy access to Internet-connected computers. Mobile interventions offer several benefits compared to in-person approaches. First, they enable scheduling flexibility and decrease scheduling burdens, thus facilitating accessibility and compliance with intervention requirements and ultimately increasing cost-effectiveness [27]. Second, digital technology can enrich the quality of treatment by incorporating innovative methods of communication, and by making treatment adaptive and responsive to dynamic, ecologically valid, real-time data [28]. Third, mobile interventions delivered in real-time may be accessed with greater frequency than in-person treatment approaches for brief therapeutic interactions that help consolidate support and maintain inter-session continuity [29]. Fourth, delivering the intervention in real-world settings may support the retention, reinforcement and successful generalization of trained skills [30]. Finally, mobile interventions can include opportunities for remote social engagement, like text-based motivation coaching from trained therapists or social networking via direct peer-to-peer messaging [30].

Guided by these principles, we designed Creating Live Interactions to Mitigate Barriers (CLIMB), a mobile psychosocial intervention that aims to enhance social functioning in people with CPD. CLIMB consists of the following two treatment components: (1) computerized social cognition training (SCT) exercises, and (2) optimized remote group therapy (ORGT). In line with the principles of IPT and CET, we opted for a hybrid approach, blending a structured training of social cognitive abilities (SCT) with an intervention that combines weekly group teletherapy with group texting (short message service, SMS) (ORGT). The principal goal of CLIMB is to enhance social functioning by driving improvements in social cognition, quality of life and clinical symptoms. However, in this open-label pilot study, our main objective is to validate the feasibility of the mobile intervention approach in people with CPD. Over the course of 12 months, we delivered CLIMB for 6 weeks via loaned tablets (iPads) to 27 participants recruited remotely from various locations in the United States and Canada. We evaluated adherence with study procedures, attrition rates, engagement metrics and acceptability of the intervention. In addition, we explored the effects of 6 weeks of CLIMB on measures of social cognition, quality of life and clinical symptoms. Finally, we explored possible predictors of engagement with treatment components, and examined whether engagement influenced changes in outcome measures.

Methods

Intervention Design

CLIMB consists of novel treatment components consisting of SCT exercises and ORGT.

Social Cognitive Training Exercises

The SCT computerized exercises used in the study are a subset of the social cognitive training suite called SocialVille, developed by Nahum et al, which aims to treat social cognition deficits targeting the impaired brain systems underlying social cognition [31]. The rationale for the training exercises has been reported elsewhere [31]. Briefly, the exercises harness the principles of brain plasticity, employing speeded, accurate and increasingly more challenging discriminations of socially-relevant information (eg, eye gazes, emotional faces, prosody, social situations). Participants progress through each exercise in a defined order of difficulty, generally moving from more simple levels (eg, easy to discriminate stimulus types, less response options) to more complex levels (eg, greater rule complexity, greater similarity between stimuli, etc). A single-arm open-label feasibility study of SocialVille delivered remotely to a small sample of young adults with psychosis found high adherence with the training requirements, and significant improvements on untrained measures of social cognition, social functioning, motivation and reward sensitivity [31].

The SCT exercises chosen in CLIMB target gaze perception, visual emotion perception, prosody, theory of mind, affective memory and attribution bias, as these core domains of social cognition are pervasively impaired in CPD and underlie most critical factors of real-world functioning including decreased quality of life and poorer community and occupational functioning [1,11-14]. A full description of the exercises is provided in Multimedia Appendix 1. The exercises are personalized, in that (1) the level of training difficulty and progression for each exercise is individually adaptive to ensure that each user is appropriately challenged; (2) although the total number of levels to be completed for each exercise is fixed and equal for all participants, they can choose any 4 of the 7 exercises to complete on every session; (3) participants can dynamically set the desired number of sessions to complete every week and monitor their progress in real-time; and (4) summary screens including game metrics (points) and exercise metrics (progress) are shown to the participant at the end of each level.

Optimized Remote Group Therapy

ORGT is an innovative integrated treatment approach that uses mobile technology to implement weekly group teletherapy sessions with group texting. In line with current recommendations for group therapy in CPD, individuals are grouped into cohorts of 3 to 6, within a similar age range (within 5 years) [32,33]. A master’s level clinician leads the group teletherapy sessions and a moderator assists participants during ORGT by facilitating and organizing the sessions and by leading the group text chat.

Group Teletherapy Sessions

Participants attend weekly, 60-minute group teletherapy sessions. Sessions are based on recovery-model principles [34] and on the Raise Early Treatment Program Manual [35]. Prior to beginning the first group therapy session, two online surveys that assess current social difficulties and preference for topics to be discussed during the group teletherapy sessions are administered. The clinician evaluates data from these surveys to familiarize herself with the patient-centered goals. In the first session, after introductions, the clinician teaches Specific, Meaningful, Agreed Upon, Realistic, Timely (SMART) goals [36]. For every session, participants set a SMART goal appropriate to their level of recovery. The following sessions start with an initial check-in (approximately 10 minutes) where participants report on the SMART goal that they attempted during the week. This is followed by psycho-education and a discussion of shared experiences (approximately 15 minutes). Here, the clinician lets participants pick a topic. The topics covered include how to make friends, how to improve social skills, how to improve motivation, how to identify a relapse and how to succeed in a job or at school. Participants discuss the topic while the clinician moderates the conversation and invites participation by all members of the group. The clinician also contributes by sharing information, such as feedback on the importance of reciprocity in social relationships or ways to motivate yourself with rewards. The next segment of the session is dedicated to learning or practicing a skill (approximately 15 minutes). The skills covered include a variety of mindfulness techniques, social skills training and relapse prevention planning. Finally, participants set a personalized SMART goal for the upcoming session (approximately 10 minutes).

Group Text Chat

The group text chat is used between video calls to optimize group teletherapy by maintaining inter-treatment session continuity with participants, by helping engage them in study procedures and by offering more opportunities for social engagement and peer support. The clinician and the moderator use the group text chat to (1) supplement group teletherapy sessions by sending links and articles about information and topics discussed during the video calls; (2) notify the group of study updates; (3) remind the group of scheduled sessions and training requirements; and (4) message participants for remote technical support and solution-focused problem-solving. The moderator encourages participants to use the group text chat to share personal artistic projects with the group through links, videos, pictures, drawings, poems and quotes. The clinician and moderator promote a non-stigmatizing approach to psychotic experiences.

Feasibility Trial

Recruitment and Enrollment

For the pilot feasibility trial, study participants were recruited online: information about the study aims and procedures was posted on Craigslist, Reddit, the National Alliance on Mental Illness newsletter and our study website. Interested individuals contacted research staff via phone call, text or email. During the initial phone screening, study personnel verified that potential participants met the following inclusion criteria: (1) prior clinical diagnosis of schizophrenia, schizoaffective disorder or bipolar disorder with psychotic features, (2) age 18 to 65 years, (3) no neurological disorder or history of traumatic brain injury, (4) no substance dependence or serious substance use in the past 6 months, (5) current treatment with a mental health care provider, (6) no hospitalization in the last 3 months and no changes in psychiatric medications for at least 1 month, (7) visual, auditory and motor capacity to use an iPad, and (8) access to an Internet-connected device with a camera/webcam and an active email account. This last criterion was required in order to sign the consent form digitally and to undergo the eligibility diagnostic interview before iPad shipment.

After the initial phone screening, study personnel provided informed consent documents to participants remotely using Qualtrics (Provo, Utah, USA). Following the initial screening and consent, participants underwent a diagnostic evaluation using the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders-IV Text Revision (DSM-IV-TR) (SCID) [37], which was administered remotely by graduate-level psychology students, using the Health Insurance Portability and Accountability Act (HIPAA) compliant video-calling software Vsee (Vsee Lab, LLC, Sunnyvale, California, USA). Participants who met DSM-IV-TR criteria for schizophrenia, schizoaffective disorder or bipolar disorder (psychotic disorders) with psychotic features were enrolled in the study.

Study Procedures

The study procedures are depicted in Figure 1. After enrollment, participants were given the opportunity to use their own iPad or to receive a study iPad via the mail that will be loaned to them for the duration of the trial. Study apps were preinstalled on the loaned iPads before shipment. Next, a one-on-one phone call was arranged to orient participants to the various apps. Enrolled participants were then placed on a waiting list and as soon as at least 3 individuals of similar age were enrolled and ready to participate in the trial, the cohort was assembled and underwent the assessment battery remotely using iPads. Once all cohort participants completed the assessment battery, they were engaged in the intervention for 6 weeks.

ORGT was delivered through Google Hangouts, a free platform that offers group-based videoconferencing, text chats and multimodal file sharing. At the beginning of the intervention, the moderator created a Hangouts group open to the clinician, moderator and cohort participants, and contacted the participants on the group text chat to explain her role. In the group text chat, the moderator introduced each participant to each other and to the clinician, informed them of the privacy practices in the apps and encouraged them to be respectful of their peers’ privacy. Next, the moderator invited participants to attend weekly, 60-minute group teletherapy sessions conducted through Google Hangouts video calls. In order to find a time that worked for everyone, the clinician, moderator and cohort participants indicated their availability using an online poll. During the 6 weeks of the intervention, the moderator was available online 8 hours a day, and kept the chat active in between sessions by contacting the group on a daily basis. Finally, all Hangouts interactions were archived and securely saved on Research Electronic Data Capture (REDCap), a HIPAA-compliant database.

The SCT program was delivered through the BrainHQ-Research app and was provided free of charge by Posit Science Inc. Participants were encouraged to complete 18 hours of SCT over the course of 6 weeks, preferably for 3 hours a week. In each training session, participants engaged with 4 different exercises, performing each exercise for about 7.5 minutes. To access the SCT program, participants logged into BrainHQ-Research using a unique study-provided login that contained no personally identifiable information. The moderator tracked user performance and treatment compliance remotely using a secure online portal. Weekly one-on-one phone calls were scheduled to discuss with participants their compliance with SCT requirements. Individualized motivational interviewing techniques were used when necessary to increase training frequency and structure training schedules [38]. For participants who completed less than 1 hour of SCT in the previous week, check ins were intensified up to 3 times a week. While in the intervention, participants continued to receive treatment by their outside providers (eg, psychoeducation, psychotherapy, adjustments in medications as clinically indicated).

Within 1 week after finishing the intervention, participants were asked to complete the assessment battery on iPads. Finally, they were asked to fill out an online exit survey in which they rated enjoyment, ease of use, perceived benefits and ease of fit into daily schedule.

If participants returned loaned iPads undamaged and fully functional, they were provided monetary compensation for participating in the study via a mailed check. Participants who completed all study procedures successfully earned US $285. Participants were paid US $5 for each completed hour of SCT and each ORGT video call attended, and US $15 per hour for assessments. Participants were not compensated for their participation in the group text chat.


Study participants were recruited online from various platforms and websites. After the phone screening, they signed informed consent documents digitally using Qualtrics. Next, they underwent a remote diagnostic evaluation (SCID), which was administered using the video-calling software Vsee. Once enrolled, participants received via mail an iPad, through which they completed an assessment battery. Next, they engaged in the CLIMB intervention for 6 weeks. CLIMB consists of a SCT program and ORGT. Within 1 week after finishing the intervention, participants completed the assessment battery on the iPad. Finally, they filled out an online exit survey. Upon the redelivery of the loaned iPad, they received compensation via check.

Assessments

Social Cognition

We assessed social cognition by means of the Prosody Identification Task (PROID) [39], and the Bell-Lysaker Emotion Recognition Test (BLERT) [40], two well-validated computerized tasks. These tasks measure two of the constructs that were trained, but distinct and independent from the specific SCT exercises used in the intervention. PROID is a vocal identification task that assesses a subject’s ability to perceive and discriminate emotion in the speech of others. The test consists of 21 sentences of neutral content that are spoken aloud by male and female speakers to convey 1 of 7 different emotions (happiness, sadness, anger, fear, surprise, disgust and no emotion), as well as utterances with no emotional expressions. Participants first identify the emotional expression of each utterance and then rate the utterance's emotional intensity on a scale from 1 (very low intensity) to 9 (very high intensity). BLERT is an affect recognition task consisting of 21 short video clips in which an actor performs 1 of 3 dialogues while portraying 1 of 7 different emotions. Participants choose which of the 7 emotions listed on the screen best describes the affective quality enacted by the actor. For both tasks, trial-by-trial accuracy data are factorized for each emotion separately, and an overall accuracy score is also provided. iOS versions of BLERT and PROID were developed using original stimulus sets provided by the authors, embedded in the BrainHQ-Research app and administered without remote supervision. Alternate forms of PROID and BLERT were counterbalanced before and after the intervention.

Quality of Life

Quality of life was evaluated by means of the Schizophrenia Quality of Life Scale (SQLS) self-report questionnaire. SQLS is a 30-item questionnaire that requires a 7-day retrospective self-assessment of quality of life [41]. Results are scored using a 5-point Likert-type scale ranging from 1 (“Never”) to 5 (“Always”). Total score ranges from 30 (best status as measured on the SQLS) to 150 (the worst status as measured on the SQLS). The scale comprises the “Psychosocial,” “Motivation and Energy,” and “Symptoms and Side-Effects” subscales, with the purpose of indicating the extent of difficulty on each domain. The “Psychosocial” subscale (15 items) addresses various emotional problems, for example, feeling lonely, depressed or hopeless, as well as feelings of difficulty mixing in social situations and feeling worried about the future. The “Motivation and Energy” subscale (7 items) addresses various problems of motivation and activity, such as lacking the will to do things, while the “Symptoms and Side-Effects” subscale (8 items) addresses issues such as sleep disturbance, blurred vision, dizziness, muscle twitches and dry mouth, which can be caused by medications. The SQLS was digitized and completed without supervision through iPads using Qualtrics.

Clinical Symptoms

Clinical symptoms were assessed using the Positive and Negative Syndrome Scale (PANSS) [42]. From the PANSS scores, 6 symptom dimensions were derived: Positive, Negative, Disorganized, Excitement, Depression and Anxiety, and Other [43]. PANSS were conducted over Vsee by graduate-level psychology students trained on manual assessment procedures and observed by expert clinical supervisors. A large body of literature suggests that assessment via videoconferencing in patients with CPD is equivalent to in-person and is tolerated and well-accepted [44].

Data Analysis Plan

To investigate the feasibility of CLIMB, descriptive statistics for recruitment, enrolment and retention rates, successful completion of remote assessments and iPad return rate were examined. Based on previous studies, we hypothesized that at least 75% of enrolled participants would complete the intervention [31], and that at least 85% of devices would be returned undamaged [45].

To investigate engagement in CLIMB, descriptive statistics for hours of SCT completed over the course of 6 weeks, attendance rate for group teletherapy and number of group chat messages and words were examined. Based on previous studies, we hypothesized that participants would complete at least 1 hour of SCT per week [31], participate in 80% of the group teletherapy sessions [22], and send at least 8 messages per week [30].

To investigate the acceptability of CLIMB, descriptive statistics from the CLIMB exit survey ratings for overall enjoyment, ease of use, ease of fit into daily schedule and perceived benefits were examined. We hypothesized ratings of at least 3 on the 5-point Likert scale items [31].

To explore the initial effects of CLIMB on study outcomes, we performed analyses on data obtained from all enrolled participants (N=27). Because we were interested in examining the ecological feasibility of CLIMB, we did not restrict the analyses only to participants who adhered to all intervention recommendations. Post-intervention data were not collected on 6 enrolled participants who dropped out at various stages of the intervention (see Figure 2). All outcome variables were normally distributed. Pre- to post-changes in outcome measures were examined using paired sample t tests. Within-group effect sizes (Cohen’s d) were computed using the mean change scores (post-treatment minus baseline) and the change score standard deviations. Because there were inter-individual differences in terms of engagement with each treatment component, and some engagement metrics were not normally distributed, we used non parametric correlations to test whether (1) engagement metrics were correlated; (2) demographic variables, symptoms or quality of life correlated with engagement metrics; and (3) changes in outcome scores correlated with engagement metrics. In the cases of significant pre- to post-changes and/or significant associations of these changes with engagement metrics, we used repeated measures linear mixed modeling with a diagonal covariance structure to determine whether changes in outcome measures were influenced by engagement with the intervention. Given the study attrition rate (22%), maximum likelihood (ML) estimation was used. Because we had baseline measurements for all participants, and the amount of missing data was relatively modest, it is likely that the missing at random assumption for ML was met, suggesting that it is unlikely the model results would have significantly changed had dropouts been able to be followed.

Figure 2. Consolidated Standards of Reporting Trials (CONSORT) flow diagram for Creating Live Interactions to Mitigate Barriers (CLIMB).

47 screened

screening and enrollment



11 (23%) did not sign the consent form

a (17%)

were consented, outdid not do the diagnostic interview


assessments


intervention


21 (45 %) completed

Results

Feasibility of Study Procedures

The Consolidated Standards of Reporting Trials (CONSORT) diagram of the study is shown in Figure 2. A total of 47 volunteer participants from 31 different states passed the phone screening over the course of 12 months (June 2015 to June

All phone-screened participants reported being comfortable using an iPad; 85% (40/47) had easy and regular access to wireless fidelity (WiFi), 72% (34/47) had a desktop or laptop computer in their home and 70% (33/47) had their own mobile phone with app capabilities (smartphone). The demographic information and access to mental health services for enrolled participants (N=27) is displayed in Table 1 and the geographic distribution of enrolled participants is shown in Figure 3.

JMIR MENTAL HEALTH

Biagianti et al

Table 1. Demographic and baseline clinical and functional characteristics for the enrolled participants (N=27).

Characteristic

Mean (SD) or n (%)

Sex, n (%)

Female

10 (37%)

Male

17 (63%)

Age (years), mean (SD)

28.1 (6.4)

Duration of illness (years), mean (SD)

7.0 (4.4)

Number of previous hospitalizations, mean (SD)

4.6 (4.0)

PANSSa total score, mean (SD)

59.2 (15.6)

SQLSb total score, mean (SD)

82.0 (15.9)

Education, n (%)

Drop-out in high school

1 (4%)

High school degree

4 (15%)

Currently pursuing a college degree

11 (41%)

Drop-out during college

8 (30%)

College degree

3 (11%)

Medications, n (%)

Taking antipsychotics

25 (93%)

Unmedicated

2 (7%)

Diagnosis, n (%)

Schizophrenia

9 (33%)

Schizoaffective

15 (56%)

Bipolar disorder with psychosis

3 (11%)

Access to mental health services, n (%)

Seeing a psychiatrist

18 (67%)

Seeing a case manager or nurse practitioner

7 (26%)

Seeing a psychotherapist

7 (26%)

aPANSS: Positive and Negative Symptoms Scale. bSQLS: Schizophrenia Quality of Life Scale.

Of the enrolled individuals, 4 (15%, 4/27) already had iPads. Consequently, iPads were shipped to 23 participants. Of those, 21 (91%, 21/23) were returned undamaged and fully functional, 1 (4%, 1/23) was initially withheld by an individual who dropped out of the study (and was retrieved with the help of the local police) and the final 1 (4%, 1/23) was never returned and

was rendered unusable through remote deactivation. The administration of SCID interviews over Vsee was completed successfully with all enrolled participants. Of the enrolled individuals, 26 (96%, 26/27) completed the baseline assessment battery on iPads successfully, providing valid BLERT, PROID, SQLS and PANSS data.


Engagement With Social Cognition Training and Optimized Remote Group Therapy

Participants were asked to complete 18 hours of SCT over the course of 6 weeks, but showed highly variable engagement, with a median of 9.5 hours of SCT completed (semi interquartile range of 6.3). In total, 6 participants (22%, 6/27) completed greater than 16 hours, requiring only brief weekly check ins; 8 (30%, 8/27) required regular weekly check ins and additional text reminders, and completed at least 6 hours; 8 (30%, 8/27) required intensive monitoring multiple times a week and trained 1 to 5 hours; and 4 (15%, 4/27) trained less than 1 hour and eventually dropped out, in spite of intensive monitoring. In addition, 46% (12/26) of participants completed less than 1 hour a week of training during the intervention. For a distribution of SCT hours, see Multimedia Appendix 2.

Participants attended on average 5.2 (SD 2.0) ORGT teletherapy sessions over the course of 6 weeks. Therefore, the average attendance rate for ORGT video calls was 84% (SD 28%). The participants’ self-assessment of social difficulties and preference about topics to be discussed during the group teletherapy sessions are shown in Multimedia Appendices 3 and 4. More than 40% of the participants endorsed lack of energy, social isolation, social and emotional withdrawal and general expectancy of failure. The topics that ranked as very interesting by at least 50% of the participants were, in order of preference, improving social engagement, improving speech activity, improving social cognitive skills, learning about identification of stressors and training problem solving skills.

Engagement with the ORGT group text chat was variable. Over the 6 weeks of the intervention, the total number of messages posted in the group chat by all cohort users (moderator and participants) averaged across all cohorts, was 1201 (SD 2013). The median number of messages posted by each participant per week across all participants, was 5.2, with a semi interquartile range of 12.8. The median number of words sent per participant per week was 37, with a semi interquartile range of 200. The median length of a message was 9.4 words, with a semi interquartile range of 5.1. We also calculated the ratio of moderator messages to participant messages. For instance, a ratio of 2:1 would mean that for every 2 messages sent from the moderator to participants in the group chat, each participant would post 1 message. When averaging across cohorts, we found a mean ratio of 0.97:1 (SD 0.30). The ratio reflects similar degrees of contribution to the group chat from the moderator and participants.

The attendance rate for group teletherapy positively correlated with hours of SCT completed (r=.484, P=.01), and with average number of words per message per participant (r=.44, P=.04). However, hours of SCT did not correlate with any ORGT messages and/or words metrics (all P values greater than .20).

At a qualitative level, we observed a wide range of contributions to ORGT: approximately 30% of participants were proactive during the group teletherapy sessions and sent text messages multiple times a week to the group chat, showing curiosity and appreciation, and engaging other participants in dynamic interactions; approximately 30% contributed only after encouragement from the clinician/moderator during the group teletherapy sessions and in the group text chat; and approximately 20% showed minimal contribution during the group teletherapy sessions, and left most engagement attempts in the group text chat unaddressed. Finally, original multimedia content was created by participants during the intervention and shared through Hangouts [46,47].

Acceptability of Creating Live Interactions to Mitigate Barriers (CLIMB)

Upon study completion participants completed an online exit survey to rate their experience with CLIMB. The first component of the survey was a 23-item questionnaire that asked participants to indicate how much of the time they felt that each statement was true, using a 5-point Likert scale with 1 corresponding to “none of the time” and 5 to “all of the time.” Items were grouped into 4 categories, and the following averaged ratings were obtained: (1) Enjoyment/Satisfaction had a rating of 2.99 (SD 1.09), where 3 corresponds to half of the time; (2) Program Clarity/Ease of Use had a rating of 4.18 (SD 0.90); where 4 corresponds to most of the time; (3) Ease of Fit had a rating of 2.91 (SD 1.20); and (4) Perceived Benefits had a rating of 3.25 (SD 1.18). The complete list of items is included in Multimedia Appendix 5.

Finally, participants indicated what they liked best about the program and what kept them from adhering to CLIMB according to the recommended schedule. In summary, participants enjoyed participating in the ORGT group teletherapy sessions because of the positive, non-stigmatizing experience of social support from staff and peers during the sessions. They also appreciated being able to access the intervention from home, and speaking with other participants from a safe and protected environment. Internet technical difficulties, symptom exacerbation, low perceived value of the treatment, motivational deficits, employment burden and lack of time were self-reported as reasons for low engagement with CLIMB.

Exploratory Outcomes

Significant improvements were found in pre- to post-measures of identification of vocal emotional prosody for happiness (P=.001) and happiness intensity (P=.04), as indexed by PROID (see Table 2). Participants also showed significant improvements in their ability to detect anger in a video vignette (P=.04), as indexed by BLERT. For these outcomes, effect sizes were large (d=.60 to d=.86). Trend-level improvements were observed on the SQLS total score, and “Psychosocial” (P=.09) and “Motivation and Energy” (P=.06) subscales. No significant changes were observed for PANSS ratings.

Table 2. Pre- to post-changes and effect sizes for outcome measures in study completers (N=21).

Outcome measure

Baseline, mean (SD)

Post, mean (SD)

Paired samples, t (sig)

Effect size, d

PROIDa accuracy in detecting, %

Happiness

0.35 (0.20)

0.54 (0.25)

-4.06 (0.00)

.86

Happiness intensity

0.37 (0.18)

0.52 (0.27)

-2.26 (0.04)

.68

Overall

0.58 (0.14)

0.59 (0.14)

-0.63 (0.54)

.11

BLERTb accuracy for detection (%)

Anger

0.78 (0.30)

0.92 (0.15)

-2.26 (0.04)

.60

Overall

0.72 (0.18)

0.80 (0.19)

-1.60 (0.13)

.42

SQLSc

Psychosocial

44.59 (9.58)

40.88 (10.71)

1.83 (0.09)

.36

Symptoms and side effects

18.65 (5.01)

18.18 (4.49)

0.48 (0.64)

.10

Motivation and energy

20.12 (3.55)

18.76 (3.70)

2.04 (0.06)

.37

Total score

83.35 (16.22)

77.82 (17.14)

1.76 (0.10)

.33

PANSSd

Negative symptoms

2.33 (1.02)

2.63 (0.98)

-2.07 (0.09)

-.29

Total score

60.81 (16.23)

61.05 (9.97)

-0.08 (0.94)

-.02

aPROID: Prosody Identification Task. bBLERT: Bell-Lysaker Emotion Recognition Test. cSQLS: Schizophrenia Quality of Life Scale. dPANSS: Positive and Negative Symptoms Scale.

Less severe negative symptoms at baseline correlated with total number of words (r=-.497, P=.022) and messages (r=-.479, P=.028) posted per participant on the group text chat during the 6 weeks of the intervention. In addition, total number of words posted per participant on the group text chat correlated with shorter duration of illness (r=-.497, P=.022). Post-intervention outcomes were not inter-correlated. We found positive associations at trend level between hours of SCT completed and gains in total SQLS (r=.448, P=.071), SQLS “Psychosocial” (r=.455, P=.067) and SQLS “Motivation and Energy” (r=.460, P=.063). The results from the linear mixed models are shown in Multimedia Appendix 6. Hours of SCT did not have significant effects on the pre- to post-changes in the linear mixed models.

Discussion

Principal Findings

In this study, we tested the feasibility of CLIMB, a mobile psychosocial intervention designed to enhance social functioning in people with CPD. Using Internet-connected iPads, we delivered CLIMB for 6 weeks to people with CPD recruited remotely from 31 locations throughout the United States and Canada. We found that CLIMB is a highly feasible intervention with high enrolment, retention, iPad return and remote assessment completion rates. In particular, the attrition and device return rates in our study are similar to other studies testing mobile phone apps in patients with psychotic disorders [29,45]. Taken together, these findings indicate that delivering assessments and treatments remotely to people with CPD using mobile platforms is highly feasible.

In line with other mobile interventions for serious mental illness[48], engagement with the CLIMB treatment components was variable. The attendance rate for ORGT group teletherapy sessions was high (84%, SD 28%), and comparable to that of in-person group therapy approaches [22]. Despite significant efforts to monitor, support and keep participants engaged in SCT, 46% (12/26) of participants completed less than 1 hour a week of training during the intervention. One possible explanation of this differential engagement is that completion of SCT exercises requires sustained effort, focused attention and active planning and engagement by the user, whereas participation in ORGT may be less effortful, more directly rewarding and require less actively focused attention. Similar to previous studies [30], participants engaged in ORGT group text chatting multiple days a week. There were, nonetheless, inter-individual qualitative differences in terms of contribution such that some individuals engaged actively in text-based conversations with the group and shared original content (short videos, pictures, drawings, poems), while the majority of the users contributed marginally or minimally to the group text chat. This heterogeneity may partially be explained by the fact that contributions to the group text chat were not compensated. Baseline characteristics likely account for these inter-individual differences, where we found that participants with shorter duration of illness and fewer negative symptoms posted more words in the group text chat, though they did not complete more hours of SCT or showed higher attendance rate in the group teletherapy sessions. These findings suggest that the group text chat is a desired modality of social engagement particularly for individuals recently diagnosed with a psychotic disorder who have less severe negative symptoms. Interestingly, the higher attendance rate in ORGT group teletherapy sessions was associated with more hours of SCT and longer messages in the group text chat, whereas hours of SCT did not correlate with group text chat metrics, possibly suggesting that engagement in remote video calls had a positive effect on engagement in the other components of treatment.

The overall acceptability of CLIMB was medium, as reflected in the satisfaction ratings endorsed in the exit survey about ease of use and perceived benefits, as well as the retention rate. Participants valued the ease of use of the CLIMB apps, the ability to access the intervention from a safe and protected environment, and they felt comfortable and accepted when sharing subjective experiences with staff and other participants. Finally, 5 (19%, 5/27) participants provided unsolicited qualitative feedback about their participation (for representative excerpts, see Multimedia Appendix 7). Qualitative and quantitative data about acceptability collected from study participants in this pilot will inform future iterations on intervention development and optimization to meet the expectations and preferences of prospective service users.

We also examined the effect of 6 weeks of CLIMB on outcomes. We found significant pre- to post-intervention improvements on specific aspects of social cognition (prosody identification of happiness and recognition of anger) with large effect sizes, and trend-level improvements on quality of life self-reports, with medium effect sizes for the “Psychosocial” and “Motivation and Energy” subscales. Although there are currently no rigorous studies testing remote SCT in CPD, our within-group effect sizes for social cognition outcomes are comparable to recent reports of in-person SCT [16]. Improvements in quality of life self-reports showed trend-level relationships with hours of SCT completed. An appropriately powered, randomized controlled trial is required to determine whether CLIMB induces improvements in social cognitive outcomes, clinical symptoms, and quality of life.

Limitations

Our study had several limitations. We recruited exclusively using the Internet, which most likely biases the sample, making it not representative of the larger population of individuals living with CPD. More than 70% of participants who passed the phone screening had a mobile phone, a desktop and/or laptop computer in their home and regular access to WiFi, whereas a recent meta-analysis found that mobile phone ownership among people with CPD was only 35% [49]. However, several lines of evidence now indicate that people with CPD already use their mobile devices to manage their illness and promote their recovery, and that mobile devices are an acceptable mental health method to deliver innovative interventions to people with CPD. For example, in surveys of mobile interventions acceptability, over half of the patients with CPD responded in favor of using mobile devices for tracking and/or monitoring their mental health, and for facilitating patient contact with health professionals [49]. As well, a large sample of individuals with CPD surveyed online reported that Web-based technology helped with identifying coping strategies and connecting for support with family and friends [50]. In addition, a convenient sample of young people recruited from two specialized early intervention programs for psychosis manifested interest in using the Internet, social media and mobile technologies for receiving mental health-related services [51]. Finally, a recent comprehensive review found high acceptability of delivered online and mobile interventions for serious mental illness, particularly when participants were provided remote online support, with the majority of studies reporting no effects of age, sex, educational level or clinical characteristics on acceptability [48]. While these findings provide evidence that acceptability is unlikely to represent a barrier to the implementation of CLIMB in mental health care settings, more problematic is the generalizability of the findings about intervention use, as less tech-oriented individuals with CPD may still find the approach acceptable, but not sufficiently engaging.

Two other factors contributed to the sampling bias. In order to be enrolled in the study, participants had to have awareness of their illness and be receiving mental health care. In our sample, 93% (25/27) of study participants were taking antipsychotic medications and 67% (18/27) were seeing a psychiatrist. These rates of mental health care use are much higher than the general CPD population in the United States, where at least 40% of people with CPD have no contact with mental health services [52]. As CLIMB has been designed to meet the needs of those who are unable or unwilling to come in to mental health clinics to access treatments for social functioning, future research will seek to recruit a mixed sample of service users directly drawn

from diverse community-based settings and non-service users in order to examine engagement patterns on a more representative sample of patients with CPD. As inadequate engagement may detrimentally impact efficacy, dose-response studies will also be conducted to determine the necessary intervention duration and intensity to drive meaningful improvements in outcomes.

While it is true that the high recruitment and retention rates found in the study are likely the result of the sampling bias that may limit the generalizability of the findings and the ability to draw conclusions about the scalability potential of CLIMB, our sample nonetheless reflects a segment of younger individuals with CPD who are technology oriented, actively engaged in their recovery and treatment and for whom digital platforms seem to be a preferable intervention delivery modality [51].

Since the goal of this pilot study was to assess the feasibility of delivering CLIMB remotely and not its efficacy, we did not include a control group. We also did not place restrictions on medication regimens during study participation. Therefore, we cannot rule out non-specific effects of study participation and medication effects on the observed improvements on proximal measures of social cognition. In addition, the fact that participants were provided remuneration for each ORGT session attended and each hour of SCT completed, likely biased the data about engagement and adherence. Therefore, our results may not translate to real-world settings where this payment schedule may not be provided. Cost-effectiveness analyses and focus groups will be conducted to devise scalable solutions for sustained engagement that match users’ preferences and

Biagianti et al expectations. Finally, the current support protocol may be manageable for CLIMB moderators when serving small cohorts of patients, but may require significant adaptations to disseminate effectively.

Conclusions

We demonstrated that it is feasible to target improvements in social cognition in people with CPD using an innovative and scalable treatment package that is delivered remotely, and that combines structured training of social cognitive abilities with a group-based, multimodal, ecological psychotherapeutic intervention. Future efficacy studies will evaluate the degree of improvement in various domains, including social functioning, social cognition, clinical symptoms and quality of life.

More importantly, results from this study indicate that it is feasible and acceptable to engage people with CPD in remote assessments and treatment using mobile devices. This has important implications in terms of access, engagement and dissemination of mental health services. First, providers will be able to interact with mobile interventions to monitor patient status remotely and provide inter-session extended support, at minimal costs and without requiring local infrastructures. Second, patients living in under-resourced areas who are unable or unwilling to come in to the clinic can benefit from specialized treatment options that may not be available locally. If successful, this approach has far-reaching implications for public health. As our knowledge of how to deliver effective treatments using remote digital technology grows, we will be able to reduce disparities in mental health outcomes, and promote equity in access to mental health care.

Acknowledgments

The authors thank Kevin Delucchi for his statistical consultation. They also thank Charlie Ward, Ken Casaletto and Sophia Quraishi, members of the CLIMB lab at University of California, San Francisco (UCSF) for their dedicated efforts and contributions to achieving the reported results. Finally, the authors also thank the patients for participating in the study. This publication was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 TR000004. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Conflicts of Interest

The cognitive training software, BrainHQ-Research, used in this study was supplied free of charge by Posit Science. Dr Vinogradov is a site primary investigator on a Small Business Innovation Research (SBIR) grant to Posit Science, a company with a commercial interest in the cognitive training software used in these studies. Dr Nahum was an employee of Posit Science at the time of the study and is the developer of the SocialVille program. None of the other authors have any financial interest in Posit Science. All authors declare no other conflicts of interest.

Multimedia Appendix 1

Descriptions of the social cognitive training (SCT) exercises.

[PDF File (Adobe PDF File), 354KB - mental_v3i4e52_app1.pdf]

Multimedia Appendix 2

Distribution of hours of social cognitive training completed over the course of 6 weeks (n=26). Each bar represents a Creating Live Interactions to Mitigate Barriers (CLIMB) participant.

[PDF File (Adobe PDF File), 53KB - mental_v3i4e52_app2.pdf ]

Multimedia Appendix 3

Results from the online survey administered at baseline to assess current social difficulties.

[PDF File (Adobe PDF File), 152KB - mental_v3i4e52_app3.pdf ]

Multimedia Appendix 4

Results from the online survey administered at baseline to review preferences for topics to be discussed during optimized remote group therapy (ORGT) sessions.

[PDF File (Adobe PDF File), 150KB - mental_v3i4e52_app4.pdf ]

Multimedia Appendix 5

Quantitative data from the online exit survey. A 5-point Likert scale was used to determine how much of the time participants felt that each statement was true with 1 corresponding to none of the time, 2 to a little bit of the time, 3 to about half the time, 4 to most of the time, and 5 to all of the time (n=21).

[PDF File (Adobe PDF File), 151KB - mental_v3i4e52_app5.pdf ]

Multimedia Appendix 6

Linear mixed models for outcome variables adjusting for hours of social cognition training (SCT) and using maximum likelihood (ML) estimation (N=27).

[PDF File (Adobe PDF File), 149KB - mental_v3i4e52_app6.pdf ]

Multimedia Appendix 7

Excerpts of feedback provided by Creating Live Interactions to Mitigate Barriers (CLIMB) participants.

[PDF File (Adobe PDF File), 214KB - mental_v3i4e52_app7.pdf ]

References

Abbreviations

BLERT: Bell-Lysaker Emotion Recognition Test

CET: cognitive enhancement therapy

CLIMB: Creating Live Interactions to Mitigate Barriers

DSM-IV-TR: Statistical Manual of Mental Disorders-IV Text Revision

HIPAA: Health Insurance Portability and Accountability Act

IPT: integrated psychological therapy

ML: maximum likelihood

ORGT: optimized remote group therapy

PANSS: Positive and Negative Symptoms Scale

PROID: Prosody Identification Task

SCID: Structured Clinical Interview for the DSM-IV-TR

SCT: social cognition training

SMART: Specific, Meaningful, Agreed Upon, Realistic, Timely

SQLS: Schizophrenia Quality of Life Scale

WiFi: wireless fidelity

Edited by G Eysenbach; submitted 20.09.16; peer-reviewed by J Torous, N Berry; comments to author 12.10.16; revised version received 10.11.16; accepted 17.11.16; published 13.12.16

Please cite as:

Biagianti B, Schlosser D, Nahum M, Woolley J, Vinogradov S

Creating Live Interactions to Mitigate Barriers (CLIMB): A Mobile Intervention to Improve Social Functioning in People With Chronic Psychotic Disorders

JMIR MentHealth 2016;3(4):e52

URL: http://mental.jmir.org/2016/4/e52/

doi: 10.2196/mental.6671

PMID:27965190

©Bruno Biagianti, Danielle Schlosser, Mor Nahum, Josh Woolley, Sophia Vinogradov. Originally published in JMIR Mental Health (http://mental.jmir.org), 13.12.2016. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.


Contents lists available at ScienceDirect

Schizophrenia Research: Cognition

journal homepage: www.elsevier.com/locate/scog


Research paper

Feasibility and preliminary efficacy of remotely delivering cognitive training      cmssMaik

to people with schizophrenia using tablets

Bruno Biagiantia,b,*, Melissa Fisherc, Lisa Howarda, Abby Rowlandsa, Sophia Vinogradovc,1, Joshua Woolleya,1

a Department of Psychiatry, University of California, San Francisco, USA

b Posit Science, Inc., USA

c Department of Psychiatry, University of Minnesota, USA

ARTICLE INFO


ABSTRACT


Keywords:

Schizophrenia

Cognitive remediation

Neuroplasticity

Mobile health


Background: Limited access to Cognitive Training (CT) for people with schizophrenia (SZ) prevents widespread adoption of this intervention. Delivering CT remotely via tablets may increase accessibility, improve scheduling flexibility, and diminish patient burden.

Methods: In this reanalysis of data from a larger trial of CT, we compared two samples of individuals with SZ who chose to complete 40 h of CT either on desktop computers in the laboratory (N = 33) or remotely via iPads (N = 41). We examined attrition rates and adherence to training, and investigated whether remote iPad-based CT and in-person desktop-based CT induced significantly different improvements in cognitive and real-world functioning.

Results: The attrition rate was 36.6%. On average, participants completed 3.06 h of CT per week. There were no significant between-group differences in attrition and adherence to CT requirements. Participants who completed iPad-based CT were significantly younger and had lower symptoms at baseline compared to participants who completed CT on the lab desktops. Controlling for age and symptom severity, rANCOVA showed that iPadbased and desktop-based CT similarly and significantly improved verbal learning and problem solving. Main effects of time, at trend level significance, were evident in global cognition, verbal memory, quality of life, and social functioning. All group by time interactions were non-significant except for verbal memory, where iPad users showed greater gains. Within-group effect sizes for changes in outcomes were in the small range.

Conclusion: Although underpowered and not randomized, this study demonstrates that delivering CT remotely to people with SZ using tablets is feasible and results in retention rates, adherence, and cognitive and functional outcome improvements that are comparable to those observed when CT is delivered in the laboratory. This has important implications in terms of scalability and dissemination of CT. These results require confirmation in larger samples.

1. Introduction

Schizophrenia (SZ) is associated with a wide range of Cognitive Impairments (CIs), including deficits in attention, speed of processing, learning and memory, problem solving, and executive functioning that are present early in the course of illness and are more enduring than psychotic symptoms (Green et al., 2004). These CIs undermine independent living and are associated with decreased lifelong community and occupational functioning even when psychotic symptoms are in remission (Kurtz et al., 2008). As a result, CIs account for 20-60% of the variance in functional outcome of individuals with SZ (Green, 1996).

Discovering methods to treat CIs in SZ early in the course of illness, before functional and psychosocial deterioration has occurred, is a major goal of 21st century psychiatry. New treatment approaches to enhance cognition, both pharmacological and behavioral, have been tested for patients with SZ. To date, none of the pharmacological trials found significant effects on CIs compared to placebo (Keefe et al., 2013). However, there is growing evidence that an intensive computerized neuroplasticity-informed Cognitive Training (CT) program may be an effective treatment for CIs in SZ.

Our research group has demonstrated that CT targeting the auditory system in adults with SZ improves early dynamic imaging responses in auditory and prefrontal cortices, as well as global cognition, speed of processing, verbal learning, and verbal memory, and that such cognitive gains predict enhanced quality of life at 6 month follow-up (Biagianti et al., 2016a; Dale et al., 2010, 2015; Fisher et al., 2010, 2016; Subramaniam et al., 2012). Additionally, a new set of exercises targeting processing speed and working memory in the social cognitive domain was tested in young adults with SZ. Results from the pilot study indicated significant improvements on prosody identification, facial memory, social functioning, motivation and reward sensitivity (Nahum et al., 2014). More recently, a randomized controlled trial found that supplementing CT with these social cognitive exercises in people with psychotic disorders confers greater benefits in prosody identification and reward processing relative to CT alone(Fisher et al., 2017). While preliminary evidence indicates that CT is efficacious, access to and engagement with CT by individuals with SZ remain outstanding challenges. This prevents widespread and optimal utilization of this promising intervention. In our studies, we have asked individuals with SZ to complete, without supervision, one hour of training exercises a day, five times a week, for a period of 8-10 weeks. Therefore, this intervention can place a high scheduling burden and become untenable for those who are in school or employed, have caregiver demands, or other medical appointments to manage. Additionally, CT is currently delivered as an experimental treatment in only a few specialized mental health clinics, and may not be accessible to people who live in rural or under-resourced areas or are without transportation (Kohn et al., 2004). The time spent reaching the clinic may therefore be another barrier to the implementation of this treatment. Lastly, some individuals with SZ hesitate to approach traditional mental health treatment settings because of stigma, which interferes with help-seeking behaviors (Angermeyer et al., 2013). These factors may all account for the high attrition rates found in CT trials, with important implications on the scalability of this intervention (Biagianti et al., 2016a).

* Corresponding author at: 401 Parnassus Ave, LP-255, San Francisco, CA 94143, USA.

E-mail address: bruno.biagianti@ucsf.edu (B. Biagianti).

1 Co-senior authors.

http://dx.doi.org/10.10167j.scog.2017.07.003

Received 19 May 2017; Received in revised form 26 July 2017; Accepted 27 July 2017

Available online 03 August 2017

2215-0013/ © 2017 Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/BY/4.0/).


Advances in interactive software development and health care delivery provide a unique opportunity to overcome these limitations. The rapid expansion of mobile technology in this population (Gay et al., 2016), with 81.4% of individuals with SZ owning a mobile phone (Firth et al., 2016), has already revolutionized the field of treatment development and delivery, allowing users to engage with innovative interventions, entirely remotely, anytime, anywhere, on their own schedule (Berry et al., 2016). Internet and mobile interventions are acceptable and feasible for individuals with SZ and have the potential to improve clinical and functional outcomes (Alvarez-Jimenez et al., 2014; Biagianti et al., 2016b). Similarly, the remote delivery of CT through mobile platforms enables scheduling flexibility and decreases scheduling burden, which may improve adherence to intervention requirements and ultimately increasing cost-effectiveness (Ventura et al., 2013). Additionally, through mobile platforms, CT can be delivered to individuals with SZ who are unable or unwilling to come in to the clinic. In doing so, mobile platforms may improve access, expand reach, and target underserved vulnerable populations.

Recent evidence suggests that CT can be feasibly delivered using mobile platforms. A 4-week feasibility trial delivering 20 h of iPad-assisted cognitive remediation vs. treatment as usual to 20 first-episode SZ in-patients showed significant improvements in working memory and good acceptability and adherence (Dang et al., 2014). More recently, our group demonstrated that it is feasible and acceptable to engage individuals with SZ in social cognition training entirely remotely using iPads (Biagianti et al., 2016b). Taken together, these findings indicate that delivering treatments including CT via internet and mobile platforms is acceptable and feasible. However, given the early state of current research, no studies have evaluated whether delivering evidence-based treatments like CT using online and mobile platforms is as efficacious as in-person delivery.

In this study, we analyzed data from a cohort of participants with SZ from our parent study (described below) who had a choice of completing 40 h of CT either on desktop computers in the laboratory, or remotely on iPads. We compared attrition rates and adherence to training requirements. We also investigated whether remote iPad-based CT and in-person desktop-based CT induce significantly different improvements in cognitive and real-world functioning.

2. Methods

This is a reanalysis of data from a double-blind Randomized Controlled Trial (RCT, ClinicalTrials.gov NCT02105779) that investigated the effects of supplementing CT with social cognitive exercises, as compared with CT alone (Fisher et al., 2017). From 2010 to February 2013, the CT program was only available on desktop computers in the laboratory. This changed in March 2013 when the CT program also became available on iPad devices that could be used remotely. From that point until the study's completion date (June 2016), participants were given the option to complete CT using desktop computers in the laboratory or to participate in the intervention remotely using provided iPads. There were no differences between the iPad and the desktop versions of CT in terms of the stimulus sets, stimulus progressions, adapting parameters, or logic of each exercise. The platforms only varied with respect to the user interface and exercise graphics.

The total number of participants randomized in the original trial was N =111. In this study, however, post-hoc analyses were only performed on the subset (N = 74) of data collected from participants who enrolled in the study from March 2013 to June 2016, to allow for a more accurate comparison among participants who could choose both their device and location of use. This comparison is not based on the randomization criterion used in the parent RCT (CT + social cognitive exercises vs CT alone). Here, we directly contrast participants who freely chose to complete CT on iPads with those who chose to complete CT on desktops. Therefore, both groups include a mix of participants from the two treatment arms of the RCT.

Participants were recruited from the San Francisco VA Medical Center outpatient clinic, other local community mental health centers, and via presentations and online advertisements. Participants were clinically stable at the time of testing (no hospitalization and stable dosages of medication over the past month). Other inclusion criteria included: 1) fluent and proficient in English, 2) WTAR > 70 (Green et al., 2008), 3) sober during assessments and training, 4) no neurological disorder. Participants reported no prior cognitive remediation treatment. This study was approved by local Institutional Review Boards.

All participants gave written informed consent for the study and were compensated for their participation in all assessments. Participants were asked to complete 40 h of the computerized CT program BrainHQ, provided free of charge by Posit Science Inc. After an intake evaluation that determined study eligibility, participants underwent an in-person structured diagnostic clinical interview and a battery of cognitive and clinical in-person assessments, which were administered again after training. For participants who completed CT in the laboratory, staff exposure during the intervention was kept to a minimum: staff aided all subjects to start each session but did not provide any coaching. Participants who opted for iPads were shown how to use the app BrainHQ in the lab before they returned home to engage in CT. Weekly phone calls were used to check in, monitor engagement, and offer technical support. Progress with CT was monitored remotely through the BrainHQ research portal. All participants received the same total number of hours of training and contact with research personnel. Multiple safeguards against loss of property were in place, including GPS tracking of the iPad and the option to remotely lock it, if lost or stolen. Security of electronic data was ensured at the level of the server, the devices, and the database.

All participants were compensated for their assessments and received $5 for each hour of CT and a $50 bonus at study completion. Payment was contingent on study participation but not performance. All patients did not have any identifiable motivation to feign or exaggerate symptoms. Patients were not using results of neuropsychological evaluations for disability compensation.

2.3. Cognitive Training (CT) program

The CT program consisted of auditory processing training exercises and auditory social cognition exercises that have been described in detail previously (Adcock et al., 2009; Nahum et al., 2014). In the auditory exercises, participants are driven to make progressively more accurate distinctions about the spectro-temporal fine-structure of auditory stimuli under conditions of increasing working memory load (i.e., increasing number of stimuli, and decreasing inter-stimulus intervals and duration of stimulus presentation). Stimuli across the exercises spanned the acoustic and organizational structure of speech, from very simple acoustic stimuli and tasks (e.g., time order judgments of rapidly successive frequency modulated sweeps) to the complex manipulations of continuous speech (e.g., narrative memory). Similarly, the social exercises harness the principles of brain plasticity, employing increasingly more challenging discriminations of sociallyrelevant stimuli, such as face identity, facial expressions, prosodic fluctuations, and gaze directions. Some exercises emphasize processing speed and require speeded discriminations, while others focus on memory, working memory, and attention load, which increase progressively during training. All exercises continuously adjust difficulty of the stimuli to an individual's performance, thus maintaining an 80%-85% correct performance rate in order to drive successful learning. We presented the auditory and social cognition modules serially over the course of the study, to avoid potential interference between the two modules. In each training session, a participant works with 4-6 exercises. Participants are asked to train for 60 min a day, 5 days a week. CT exercises are listed in Supplemental Table 1.

All neuropsychological assessments and clinical interviews were performed by highly trained raters, directly supervised by the same senior researcher (M.F.). Research staff who conducted clinical or cognitive testing first completed extensive training on testing/inter-viewing and scoring criteria of individual items (e.g., scoring videotaped sessions, observation of sessions conducted by experienced staff, and participating in mock sessions). Diagnostic assessments were administered at baseline, while all other assessments were administered at baseline and after 40 h of training.

2.4.1. Diagnostic Assessment

At study entry, each participant received a standardized diagnostic evaluation performed by research personnel trained in research diagnostic techniques. Evaluations included the Structured Clinical Interview for DSM-IV Axis I Disorders (First et al., 2002), as well as review of clinical records and interview with patient informants (e.g., psychiatrists, therapists, social workers).

The MATRICS Consensus Cognitive Battery (MCCB; (Nuechterlein et al., 2008) was administered at baseline and at training conclusion. All primary outcome measures were distinct and independent from tasks practiced during CT. In addition to the learning trials, the verbal and visual memory trials (i.e., delayed recall) of the Hopkins Verbal Learning Test-R (HVLT-R) and Brief Visuospatial Memory Test-R (BVMT-R) were administered. Alternate forms of the HVLT-R, BVMT-R, and NAB Mazes tests were administered and counterbalanced. The following domains were assessed: attention, speed of processing, working memory, verbal learning and verbal memory, visual learning and visual memory, and problem solving. All tests were scored and rescored by a second staff member blind to the first scoring. The MCCB computerized scoring program was used to compute age and gender adjusted T-scores and the composite scores. T-scores for the HVLT-R and BVMT-R delayed recall trials were computed using normative data from the published manuals.

2.4.3. Symptoms and functional outcome measures

To assess current (past 30 days) symptomatology, we used the Positive and Negative Syndrome Scale (PANSS, Kay et al., 1987). Higher scores are indicative of greater pathology. Quality of life was measured using the abbreviated Quality of Life Scale (aQLS, (Bilker et al., 2003). Social functioning was measured using the Social Functioning Scale (SFS, (Birchwood et al., 1990). Functional capacity was measured using the UCSD Performance Based Skill Assessment (UPSA, Patterson et al., 2001). Higher QLS, SFS, UPSA scores are indicative of better functioning.

2.5. Planned analyses

All variables were screened and normally distributed after Winsorizing of outlying values ( ± 2.5 SD from the mean). To investigate the feasibility and acceptability of remotely delivering CT to individuals with SZ using iPads, we compared attrition rates, hours of training completed, and training intensity between the two groups. Chi square was used to test for group differences in attrition rate. Independent sample t-tests were used to determine if there were any baseline differences between study completers with those who withdrew after baseline assessments. Fisher's Exact Test or Chi-Square Test tested for group differences in categorical variables (i.e., gender, diagnosis, medications, attrition). Bivariate correlations indexed the relationship between baseline characteristics and hours of training completed.

In order to determine whether the training delivery method (iPad vs desktop) had an effect on treatment response, we ran a per-protocol analysis on study completers only (N = 47). First, we examined differences in baseline demographic, cognitive, clinical, and functional characteristics, as well as hours of training completed and training intensity, between the two groups using Independent Samples T-tests. Repeated-measures analyses of variance were then used to compare the groups on the change in cognitive and functional outcome measures, controlling for potential baseline differences. Outcome scores collected at baseline and after training were entered as the repeated measure and training delivery method (iPad or Desktop) was entered as the between-subjects factor. Significant main effects of time accompanied by nonsignificant group by time interactions would suggest that subjects showed a significant change as a group, and that the training delivery method did not significantly influence this change. We also calculated change scores for outcome variables and explored bivariate correlations with demographics, hours of training, and training intensity.

Finally, in order to explore differences in the improvements induced by iPad-based vs. desktop-based CT, we first computed within-group effect sizes for each arm separately, by calculating the bias-adjusted standardized mean difference (Hedges' g), using the mean change scores (post treatment minus baseline) and the change score SDs. Next, we calculated confidence intervals for Hedges' g within-group effect sizes and conducted between-group comparisons to exclude statistically significant differences between desktop-based and iPad-based CT.

3. Results

Of the 74 participants who completed baseline assessments, 33 participants chose to complete 40 h of desktop-based CT condition in

Table 1

Characteristics of Participants who Completed Training via Desktop Computer and iPad.

Completed training via desktop computer (N = 21) mean (SD)

Completed Training via iPad

(N = 26) mean (SD)

T-test (p-value)

Female(N)/male(N)a

2/19

7/19

0.16

Age

49.24 (10.20)

40.38 (14.10)

2.50 (0.02)

Years of education

13.13 (2.39)

14.35 (2.51)

- 1.67 (0.10)

Wechsler test of adult

99.52 (11.40)

104.60 (9.49)

- 1.5 (0.11)

reading-premorbid IQ estimate

Diagnosisb

Schizophrenia(N)

14

17

Schizoaffective

7

8

0.84(0.66

disorder(N)

Psychosis NOS (N)

0

1

Hours of training

40.00 (0.00)

41.86 (4.95)

- 1.88 (0.07)

Training intensity (hours/

3.28 (1.28)

2.84 (1.67)

1.00 (0.33)

week)

Global cognition

27.86 (15.41)

31.18 (15.23)

- 0.73 (0.47)

Positive and Negative

69.62 (15.58)

59.29 (14.23)

2.31 (0.03)

Syndrome Scale (PANSS) total

UPSA-briefc total score

65.10 (15.46)

72.23 (14.74)

- 1.62 (0.11)

Quality of life scale-

2.62 (0.80)

3.20 (1.18)

- 1.92 (0.06)

average item rating

Social functioning scale-

104.63 (5.92)

108.14 (7.92)

- 1.65 (0.11)

average subscale

total

Statistically significant p-values are formatted in bold.

a Fisher's Exact Test results.

b Chi-Square Test results.

c University of California, San Diego, Performance-Based Skills Assessment—Brief.

the laboratory, and 41 chose to be loaned iPads and complete the intervention remotely. 12 out of 33 (36.4%) subjects from the desktopbased CT condition withdrew from the study compared to 15 of 41 (36.6%) subjects from the iPad-based CT condition, a non-significant difference (x2 < 0.001, p = 0.984). The most common reason given for dropping out was finding the demands of the training too high and being unable to make time to do the training. There were no significant differences in demographic variables, medication use or dosage, cognition, symptom severity, or functioning between those who completed the study and those who discontinued (Supplemental Tables 2 and 3). These variables were also unrelated to the number of hours completed. Participants who discontinued had significantly fewer hours per week (p = 0.02) relative to study completers (see Supplemental Table 2). All iPads were returned undamaged and fully functional.

There were no significant differences in total hours of training completed and training intensity (hours of training completed per week) between those who completed the intervention remotely on iPads and those who completed it on desktop computers in the laboratory (Table 1). In particular, iPad users completed 2.84 ± 1.67 h of training per week, compared to 3.28 ± 1.28 h in the desktop-based CT group), which suggests good adherence to the training requirements.

Participants who completed CT via iPad versus desktop computers were compared on demographic variables, baseline cognitive performance, symptoms, functional outcomes (Table 1) and medication regimens (Supplemental Table 3). All differences in baseline cognition, functioning and medication regimens were non-significant. Participants who completed iPad-based CT were significantly younger and had lower baseline symptoms compared to participants who completed CT on the lab desktops. Accordingly, repeated-measures ANCOVAs controlling for age and baseline symptom severity were used to compare the subject groups on the change in cognitive and functional outcome measures.

Omnibus test results showed significant main effects of time on Verbal Learning (p = 0.02) and Problem Solving (p = 0.05), and effects of time at trend level significance in Global Cognition (p = 0.07) and Verbal Memory (p = 0.06, Table 2). All group-by-time interactions were non-significant, except for Verbal Memory (p = 0.04). Post hoc analyses of Verbal Memory revealed non-significant improvements in the iPad participants (F = 0.45, p = 0.51), and a decrease at trend level significance in the desktop group (F = 3.47, p = 0.08). Interestingly, the effect size of the within-group change in Verbal Learning in desktop-based CT completers (g = 0.27, Table 3) was very similar to the one found in a sample of adults with persistent SZ who completed

Table 2

Scores on Cognitive and Functional Outcome Measures at Baseline, and after 40 h of Training in Participants who Completed Cognitive Training (CT) in-person on Desktops (n = 21) or Remotely via iPads (n = 26).

Desktop-based CT (N = 21)

iPad-based CT (N = 26)

Test statistics6

Baseline

After 40 h of CT

Baseline

After 40 h of CT

Main effects of time

Group by time interaction

p value

p value

Outcome measures

Mean

SD

Mean

SD

Mean

SD

Mean

SD

F(p)

F(p)

Global cognitiona

27.86

15.41

32.61

15.82

31.18

15.23

31.17

14.56

3.47 (0.07)

0.03 (0.88)

Attentiona

36.10

14.65

36.53

14.40

39.78

12.45

41.79

15.84

0.12 (0.73)

0.26 (0.61)

Speed of processinga

33.14

14.41

34.32

18.00

34.62

13.70

36.56

14.82

0.96 (0.33)

0.11 (0.75)

Working memorya

36.19

14.13

37.29

14.83

42.06

14.59

41.62

11.17

0.08 (0.97)

0.52 (0.48)

Verbal learninga

34.10

6.83

36.52

10.65

38.54

11.42

39.88

8.51

6.08 (0.02)

0.85 (0.36)

Verbal memorya

29.36

18.21

23.86

17.30

29.66

18.00

31.83

18.48

3.84 (0.06)

4.77 (0.04)

Visual learninga

38.91

17.33

39.16

13.41

36.74

14.31

38.64

14.38

1.41 (0.24)

1.12 (0.30)

Visual memorya

39.59

18.16

34.50

17.55

36.98

18.55

36.66

16.38

1.20 (0.28)

1.17 (0.29)

Problem solvinga

38.57

8.28

38.48

10.68

38.43

13.24

43.04

11.89

4.04 (0.05)

0.63 (0.43)

UPSA-briefb

65.10

15.46

65.50

17.45

72.23

14.74

74.30

14.15

0.59 (0.45)

0.00 (0.99)

Quality of life scalec

2.62

0.80

2.89

1.01

3.20

1.18

3.27

0.90

2.92 (0.10)

0.33 (0.57)

Social functioning scaled

104.63

5.92

106.92

7.04

108.14

7.92

109.19

7.14

3.62 (0.07)

0.43 (0.52)


a MATRICS Consensus Cognitive Battery (MCCB) Measures: Global Cognition (composite T-score across all MCCB measures); Attention (Continuous Performance Task-Identical Pairs); Speed of Processing (Trail Making Test Part A; Category Fluency Animal Naming; BACS Symbol Coding); Working Memory (Letter-Number Span; WMS-III Spatial Span); Verbal Learning (HVLT-R Immediate Recall); Visual Learning (BVMT-R Immediate Recall); Problem Solving (NAB Mazes). In addition to the MCCB, verbal and visual Delayed Recall from the HVLT-R and BVMT-R were administered.

b University of California, San Diego, Performance-Based Skills Assessment—Brief. Brief Total Score.

c Abbreviated Quality of Life Scale - Average Item Rating.

d Social Functioning Scale - Average Subscale Total.

e Repeated measures ANCOVA, controlling for age and PANSS symptoms, effects of time and group-by-time interaction.


Table 3

Effect Sizes and Confidence Intervals (CI) for the Within-group Changes in Cognitive and Functional Outcome Measures for Participants who Completed Cognitive Training (CT) in-person on Desktops (n = 21) or Remotely via iPads (n = 26).

Desktop-based CT (N = 21)

iPad-based CT (N = 26)

Effect size

Effect size

Outcome measuresa

ge

95% CI

ge

95% CI

Global cognitiona

0.30

(- 0.35; 0.95)

0.00

(- 0.57; 0.57)

Attentiona

0.03

(- 0.61; 0.67)

0.14

(- 0.43; 0.71)

Speed of processinga

0.07

(- 0.57; 0.71)

0.13

(- 0.43; 0.69)

Working memorya

0.07

(- 0.55; 0.69)

- 0.03

(- 0.59; 0.52)

Verbal learninga

0.27

(- 0.36; 0.89)

0.13

(- 0.43; 0.69)

Verbal memorya

-0.30

(- 0.93; 0.32)

0.12

(- 0.44; 0.67)

Visual learninga

0.02

(- 0.62; 0.65)

0.13

(- 0.43; 0.69)

Visual memorya

- 0.28

(- 0.92; 0.36)

- 0.02

(- 0.58; 0.54)

Problem solvinga

- 0.01

(- 0.63; 0.61)

0.36

(- 0.2; 0.93)

UPSA-briefb

0.02

(- 0.6; 0.65)

0.14

(- 0.43; 0.71)

Quality of life scalec

0.29

(- 0.34; 0.92)

0.06

(- 0.53; 0.65)

Social functioning

0.35

(- 0.29; 0.98)

0.14

(- 0.46; 0.74)

scaled

a MATRICS Consensus Cognitive Battery (MCCB) Measures: Global Cognition (composite T-score across all MCCB measures); Attention (Continuous Performance TaskIdentical Pairs); Speed of Processing (Trail Making Test Part A; Category Fluency Animal Naming; BACS Symbol Coding); Working Memory (Letter-Number Span; WMS-III Spatial Span); Verbal Learning (HVLT-R Immediate Recall); Visual Learning (BVMT-R Immediate Recall); Problem Solving (NAB Mazes). In addition to the MCCB, verbal and visual Delayed Recall from the HVLT-R and BVMT-R were administered.

b University of California, San Diego, Performance-Based Skills Assessment—Brief. Brief Total Score.

c Abbreviated Quality of Life Scale - Average Item Rating.

d Social Functioning Scale - Average Subscale Total.

e Within-group bias-adjusted standardized mean difference (Hedges'g), calculated using the mean change scores (post treatment minus baseline) and the change score standard deviations.

50 h of CT on desktops (g = 0.29, Fisher et al., 2016), and to the one found among subjects of the parent trial who completed CT (g = 0.27, Fisher et al., 2017). Additionally, this effect size was similar to the one reported by a meta-analysis of existing cognitive remediation approaches (g = 0.39, McGurk et al., 2007). Conversely, the effect size for iPad-based CT completers (g = 0.13) closely resembles the within-group effect size found in a study conducted in recent onset SZ (g = 0.11, Fisher et al., 2015), where participants were loaned laptop computers and completed CT remotely and without supervision.

Similarly, the effect size of the change in Problem Solving averaged across the two study groups (g = 0.18, Table 3) was similar to the one found in a sample of adults with persistent SZ who completed 50 h of CT on desktops (g = 0.12, Fisher et al., 2016), and to the one found among subjects of the parent trial who completed CT (g = 0.21, Fisher et al., 2017). However, this effect size was smaller than the one found within the group of individuals with recent onset SZ who completed 40 h of CT (g = 0.80, Fisher et al., 2015).

Omnibus test results showed main effects of time at trend level significance for the Quality of Life Scale (p = 0.10) and the Social Functioning Scale (p = 0.07), with both groups showing improvements over time (Table 2). The group-by-time interactions for these variables were non-significant. Hours of training and training intensity did not correlate with outcome change scores. For both groups, effect sizes of the within-group change in cognitive and functional outcome measures were in the small range (Table 3). Confidence intervals for effect sizes for all outcome measures overlapped between the two groups. This suggests that there were no statistically significant differences between changes induced by iPad-based CT and desktop-based CT.

4. Discussion

4.1. Feasibility and adherence

In this study, we examined the feasibility and preliminary efficacy of using tablets to remotely deliver 40 h of unsupervised computerized CT to individuals with SZ. We directly compared retention rates and adherence to the training between a sample of individuals with SZ who chose to complete CT using desktop computers in the laboratory and a sample of individuals with SZ who participated in the same intervention remotely using provided iPads. We found a non-significant between-group difference in attrition rates, suggesting that the training delivery method does not influence study retention. Additionally, we found no differences in pretreatment demographic, cognitive, clinical, and functional measures between study completers and dropouts. We also did not find baseline predictors of number of CT hours completed. Across groups, the most common reasons given for discontinuation were finding the demands of CT too high and being unable to make time to do the training. None of the iPad participants who withdrew from the study reported discomfort with using the tablet without supervision for CT.

Despite significant efforts to monitor, support, and keep participants engaged in CT, the overall study attrition rate was 36.5%. Similar attrition rates were found in the parent trial (Fisher et al., 2017)—where analyses on attrition were conducted after 70 h of training on all randomized participantsand in a study of CT in recent onset schizophrenia (Fisher et al., 2015). In these studies, some (Fisher et al., 2017) or all (Fisher et al., 2015) participants were loaned laptop computers or tablets with the necessary software and participated in the intervention remotely. However, the attrition rate found here is larger than the one found in our study of the same software in adults with persistent schizophrenia (15%, Fisher et al., 2016), where all participants performed the CT exercises in the laboratory. This suggests that delivering CT remotely via laptops or tablets may not necessarily improve study retention.

The average training intensity found in the group of iPad-based CT completers is in line with rates observed in other remote CT studies using similar versions of the program in young samples of individuals with SZ (Biagianti et al., 2016b; Nahum et al., 2014) as well as with the rates observed in our desktop CT group. We note that participants who chose to complete iPad-based CT were significantly younger and had lower baseline symptoms compared to participants who completed CT on the lab desktops. Participants who completed CT on iPads reported being comfortable using the device, liking the ability to schedule independently their training sessions and avoiding commute to engage in CT. Finally, all iPads were returned intact. Taken together, these findings indicate that delivering 40 h of CT remotely to people with SZ using tablets is feasible.

4.2. The effects of iPad vs. desktop training on cognition

The second goal of the study was to examine whether the improvements in cognitive and real-world functioning induced by iPadbased CT and desktop-based CT were different. Both groups showed significant improvements in verbal learning and problem solving, as well as improvements at trend level significance in global cognition and verbal memory. Additionally, the confidence intervals for the within-group effect sizes overlapped, indicating no statistically significant differences between improvements induced by iPad-based CT and desktop-based CT. Effect sizes for changes in outcome measures were in the small range. Inconsistent with previous reports (Fisher et al., 2015, 2016), we found a decrease in verbal memory at trend level significance in desktop-based CT completers. This decrease, along with a non-sig-nificant improvement in the iPad participants, explains the significant group by time interaction observed for verbal memory. An intent-to-treat analysis of all randomized participants from the parent study showed significant gains in a greater number of cognitive domains (i.e. global cognition, attention, speed of processing, verbal learning, visual learning, and problem solving; Fisher et al., 2017). The lack of replication in this analysis of the subset is likely due to the smaller sample size, and the use of a per-protocol analysis.

4.3. The effects of iPad vs. desktop training on social functioning and quality of life

The whole sample also showed improvements in quality of life and social functioning at trend level significance. The CT delivery method (iPad vs desktop) did not influence the magnitude of these improvements. Within-group effect sizes for these improvements were in the small range. Overlap between the confidence intervals for these effect sizes indicates no statistically significant differences between iPadbased and desktop-based CT completers. Because the omnibus test of these functional outcome measures was at trend level significance, these results require confirmation with a larger sample size. It is also possible that improvements in functional outcome measures may be dependent on the domains targeted by CT, and may also not become evident immediately after the intervention. An open-label study that tested a set of exercises targeting processing speed and working memory in the social cognitive domain in young adults early in the course of SZ found significant improvements in social functioning immediately after training (Nahum et al., 2014). In previous imaging studies of adults with chronic SZ, we found that training-induced enhancement of prefrontal activity significantly predicted improvements in quality of life six months after completion of training (Subramaniam et al., 2014), and that gains in cognition were associated with gains in functioning, but only after six months (Fisher et al., 2010).

We note that the magnitude of the effects on both cognition and functional outcome measures in the present study was small, while in our prior studies, effect sizes ranged predominantly from medium to large (Fisher et al., 2015, 2016). This difference is likely due to the fact that the computer games control condition used in our prior studies induced small, non-significant decreases in verbal cognitive measures, which contributed to the overall magnitude of the effects.

The major limitation of this study is the lack of randomization. Since the CT program became available on iPads (March 2013), all participants enrolled in the study were given the option to choose their preferred training delivery method. This implies that any difference or lack of difference between the groups might be due to self-selection. We found that the two samples were unmatched for age and baseline symptoms. Although we controlled for these variables in our analyses of variance, the sampling bias limits the generalizability of the findings. Nonetheless, we believe that the quasi-experimental nature of this approach has revealed two important aspects regarding the acceptability of the mobile delivery method for CT in this population. First, 41 participants chose iPads to access CT, whereas only 33 opted for desktops in the laboratory. This provides evidence that acceptability is unlikely to represent a barrier to the implementation of CT on mobile platforms, and is in line with a recent review of delivered online and mobile interventions for serious mental illness, suggesting high acceptability particularly when participants are provided remote online support (Berry et al., 2016). Second, those participants who chose iPads for remote training were younger and less symptomatic, possibly reflecting a segment of individuals with SZ for whom tablets seem to be a preferable intervention delivery modality (Lal et al., 2015).

Another major limitation is the small sample size, which reduced our power to detect whether improvements in outcome measures were different or equivalent between the two groups. An intent-to-treat analysis (ITT) on all subjects could have identified additional between-group differences. However, when we conducted the ITT using a linear mixed-effects model with group and time as fixed factors, model convergence was not achieved for several outcome variables, thus compromising the validity of the models' fit. Additionally, the lack of significant between-groups differences demonstrated by this study does not imply statistically equivalent efficacy, i.e., that the magnitudes of improvements are significantly similar. Equivalence and/or non-in-feriority testing methods, including the Two One-Sided Tests (TOST), require a larger sample size to statistically reject effects that fall outside the equivalence margins established for the effect size. Future studies of CT should be powered taking into account the attrition rates observed in this and other trials of unsupervised CT to draw conclusions about the comparative efficacy of these two training delivery methods.

We note that although the attrition rate found in this study is substantially higher than those found in other studies testing mobile interventions in individuals with SZ (Firth and Torous, 2015), CT - unlike most of these interventions - requires sustained effort, focused attention, and active planning and engagement by the user multiple days per week for several weeks. The high attrition rate may also be due to the fact that staff did not play an active role during the intervention, and that CT was not administered in the context of existing clinical services. These elements may explain the lower attrition rates found in similar studies of CT. For example, Fernandez-Gonzalo et al. found an attrition rate of 25% in outpatients with early illness schizophrenia who received individual coaching by neuropsychologists during all training sessions (Fernandez-Gonzalo et al., 2015). Lindenmayer et al. found an attrition rate of 7% when training was delivered to inpatients hospitalized for a 20-h per week rehabilitation protocol (Lindenmayer et al., 2013). Finally, the fact that similar retention rates were observed among those who completed the intervention on desktops in the laboratory and those who completed it remotely on tablets suggests that the high attrition is more attributable to the nature of the treatment rather than to its delivery method. Strategies that have been demonstrated efficacious at improving participation in other behavioral interventions that require ongoing commitment, like physical exercise for obesity, could help sustain engagement with CT. Examples include structured participation in groups rather than relying solely on at-home practice (Biagianti et al., 2016b; Hogarty and Flesher, 1999), behavioral economic approaches (Haff et al., 2015), motivational interviewing (Medalia and Saperstein, 2011), and gamification techniques to make the program more enjoyable (Lumsden et al., 2016). In addition to increasing study sample size, future studies should test and implement these strategies to sustain the engagement of study participants.

Other factors may limit the generalizability of the findings. First, long-term follow-up assessments are needed to determine the durability of treatment-induced gains and to evaluate whether improvements in functional outcome measures will become significant only after followup periods. Such studies are currently underway. Second, we did not place restrictions on medication regimens during study participation. Therefore, we cannot rule out non-specific effects of study participation and medication effects on the observed improvements. Third, the fact that participants were provided remuneration for each hour of CT completed likely biased data about engagement and adherence. Therefore, our results may not translate to real-world settings where this payment schedule may not be provided. Furthermore, research staff was not blind to the CT delivery method. Finally, participants in this study were well-educated, with an average age of 45, which may limit the generalizability of our results to other samples. For all these reasons, our findings should be taken with caution and replicated in future studies.

5. Conclusions

Our study suggests that remotely delivering CT to individuals with SZ using iPads is feasible, and results in retention rates and adherence to the training schedule that are comparable to those found when the same treatment is delivered in person in the laboratory using desktop computers. This has important implications in terms of access to and dissemination of CT. First, individuals with SZ living in under-resourced areas who are unable or unwilling to come in to the clinic could benefit from CT even if the treatment is not available locally. Second, through mobile platforms, providers will be able to monitor patient status remotely and provide support without requiring local infrastructures. It is important to note that the attrition rate found in this study -which seems independent of the delivery method - is likely to negatively impact treatment uptake in real-world settings. For this reason, additional research is needed to determine how to increase retention and sustain engagement. While preliminary in nature, our findings also indicate that the two training delivery methods are associated with similar improvements in verbal learning, global cognition, quality of life and social functioning, although these results require confirmation with a larger sample size.

In sum, we demonstrate here that the rapid expansion of mobile technology represents a major step forward in making CT readily available to individuals with SZ. We believe that the remote mobile approach used here to deliver CT can be successfully extended to other specialized treatment options. If so, remote digital technology will be an indispensable means to promote equity in access to mental health services and hopefully reduce disparities in mental health outcomes.

Funding source

This work was supported by National Institute of Health NCT02105779.

Conflict of interest

The cognitive training software used in these studies was supplied to the last author free of charge by Posit Science. Dr. Vinogradov is a site PI on an SBIR grant to Posit Science, a company with a commercial interest in the cognitive training software used in this studsy. Dr. Biagianti is a post-doctoral research fellow partially funded by Posit Science. None of the other authors have any financial interest in Posit Science. All authors declare no other conflicts of interest. Dr. Vinogradov serves on an advisory board for Forum pharmaceuticals.

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.scog.2017.07.003.

References

Adcock, R.A., Dale, C., Fisher, M., Aldebot, S., Genevsky, A., Simpson, G.V., Nagarajan, S., Vinogradov, S., 2009. When top-down meets bottom-up: auditory training enhances verbal memory in schizophrenia. Schizophr. Bull. 35, 1132-1141. http://dx.doi.org/ 10.1093/schbul/sbp068.

Alvarez-Jimenez, M., Alcazar-Corcoles, M.A., Gonzalez-Blanch, C., Bendall, S., McGorry, P.D., Gleeson, J.F., 2014. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr. Res. 156, 96-106. http://dx.doi.org/10.1016/j.schres.2014.03.021.

Angermeyer, M.C., Matschinger, H., Schomerus, G., 2013. Attitudes towards psychiatric treatment and people with mental illness: changes over two decades. Br. J. Psychiatry

J. Ment. Sci. 203, 146-151. http://dx.doi.org/10.1192/bjp.bp.112.122978.

Berry, N., Lobban, F., Emsley, R., Bucci, S., 2016. Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: a systematic review. J. Med. Internet Res. 18, e121. http://dx.doi.org/10. 2196/jmir.5250.

Biagianti, B., Fisher, M., Neilands, T.B., Loewy, R., Vinogradov, S., 2016a. Engagement with the auditory processing system during targeted auditory cognitive training mediates changes in cognitive outcomes in individuals with schizophrenia. Neuropsychology 30, 998-1008. http://dx.doi.org/10.1037/neu0000311.

Biagianti, B., Schlosser, D., Nahum, M., Woolley, J., Vinogradov, S., 2016b. Creating live interactions to mitigate barriers (CLIMB): a mobile intervention to improve social functioning in people with chronic psychotic disorders. JMIR Ment. Health 3, e52. http://dx.doi.org/10.2196/mental.6671.

Bilker, W.B., Brensinger, C., Kurtz, M.M., Kohler, C., Gur, R.C., Siegel, S.J., Gur, R.E., 2003. Development of an abbreviated schizophrenia quality of life scale using a new method. Neuropsychopharmacology 28, 773-777. http://dx.doi.org/10.1038/sj.npp. 1300093.

Birchwood, M., Smith, J., Cochrane, R., Wetton, S., Copestake, S., 1990. The social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br. J. Psychiatry J. Ment. Sci. 157, 853-859.

Dale, C.L., Findlay, A.M., Adcock, R.A., Vertinski, M., Fisher, M., Genevsky, A., Aldebot,

S., Subramaniam, K., Luks, T.L., Simpson, G.V., Nagarajan, S.S., Vinogradov, S., 2010. Timing is everything: neural response dynamics during syllable processing and its relation to higher-order cognition in schizophrenia and healthy comparison subjects. Int. J. Psychophysiol. 75, 183-193. http://dx.doi.org/10.1016/j.ijpsycho.2009.10. 009.

Dale, C.L., Brown, E., Fisher, M., Herman, A.B., Dowling, A., Hinkley, L.B., Subramaniam,

K. , Nagarajan, S., Vinogradov, S., 2015. Auditory cortical plasticity drives training-induced cognitive changes in schizophrenia. Schizophr. Bull. http://dx.doi.org/10. 1093/schbul/sbv087.

Dang, J., Zhang, J., Guo, Z., Lu, W., Cai, J., Shi, Z., Zhang, C., 2014. A pilot study of iPad-assisted cognitive training for schizophrenia. Arch. Psychiatr. Nurs. 28, 197-199. http://dx.doi.org/10.1016/j.apnu.2014.01.003.

Fernandez-Gonzalo, S., Turon, M., Jodar, M., Pousa, E., Hernandez Rambla, C., Garcia, R., Palao, D., 2015. A new computerized cognitive and social cognition training specifically designed for patients with schizophrenia/schizoaffective disorder in early stages of illness: a pilot study. Psychiatry Res. 228, 501-509. http://dx.doi.org/10. 1016/j.psychres.2015.06.007.

First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., 2002. Structured Clinical Interview for DSM-IV-TR Axis 1 Disorders, Research Version, Patient Edition. (SCID-I/P). Biometrics Research, New York State Psychiatric Institute, New York, NY (n.d.).

Firth, J., Torous, J., 2015. Smartphone apps for schizophrenia: a systematic review. In: JMIR MHealth UHealth. 3, e102. http://dx.doi.org/10.2196/mhealth.4930.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2016. Mobile phone ownership and endorsement of mHealthamong people with psychosis: a metaanalysis of cross-sectional studies. Schizophr. Bull. 42, 448-455. http://dx.doi.org/ 10.1093/schbul/sbv132.

Fisher, M., Holland, C., Subramaniam, K., Vinogradov, S., 2010. Neuroplasticity-based cognitive training in schizophrenia: an interim report on the effects 6 months later. Schizophr. Bull. 36, 869-879. http://dx.doi.org/10.1093/schbul/sbn170.

Fisher, M., Loewy, R., Carter, C., Lee, A., Ragland, J.D., Niendam, T., Schlosser, D., Pham,

Fisher, M., Mellon, S.H., Wolkowitz, O., Vinogradov, S., 2016. Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor. Schizophr. Res. Cogn. 3, 1-7. http://dx.doi. org/10.1016/j.scog.2015.10.006.

Fisher, M., Nahum, M., Howard, E., Rowlands, A., Brandrett, B., Kermott, A., Woolley, J., Vinogradov, S., 2017. Supplementing intensive targeted computerized cognitive training with social cognitive exercises for people with schizophrenia: an interim report. Psychiatr. Rehabil. J. 40, 21-32. http://dx.doi.org/10.1037/prj0000244.

Gay, K., Torous, J., Joseph, A., Pandya, A., Duckworth, K., 2016. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Ment. Health 3, e15. http://dx.doi.org/10.2196/mental.5379.

Green, M.F., 1996. What are the functional consequences of neurocognitive deficits in schizophrenia? Am. J. Psychiatry 153, 321-330. http://dx.doi.org/10.1176/ajp.153. 3.321.

Green, M.F., Kern, R.S., Heaton, R.K., 2004. Longitudinal studies of cognition and functional outcome in schizophrenia: implications for MATRICS. Schizophr. Res. 72, 41-51. http://dx.doi.org/10.1016/j.schres.2004.09.009.

Green, R.E., Melo, B., Christensen, B., Ngo, L.A., Monette, G., Bradbury, C., 2008. Measuring premorbid IQ in traumatic brain injury: an examination of the validity of the Wechsler test of adult reading (WTAR). J. Clin. Exp. Neuropsychol. 30, 163-172. http://dx.doi.org/10.1080/13803390701300524.

Haff, N., Patel, M.S., Lim, R., Zhu, J., Troxel, A.B., Asch, D.A., Volpp, K.G., 2015. The role of behavioral economic incentive design and demographic characteristics in financial incentive-based approaches to changing health behaviors: a meta-analysis. Am. J. Health Promot. 29, 314-323. http://dx.doi.org/10.4278/ajhp.140714-LIT-333.

Hogarty, G.E., Flesher, S., 1999. Practice principles of cognitive enhancement therapy for schizophrenia. Schizophr. Bull. 25, 693-708.

Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13, 261-276.

Keefe, R.S.E., Buchanan, R.W., Marder, S.R., Schooler, N.R., Dugar, A., Zivkov, M., Stewart, M., 2013. Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? Schizophr. Bull. 39, 417-435. http://dx.doi. org/10.1093/schbul/sbr153.

Kohn, R., Saxena, S., Levav, I., Saraceno, B., 2004. The treatment gap in mental health care. Bull. World Health Organ. 82, 858-866 (doi:S0042-96862004001100011).

Kurtz, M.M., Wexler, B.E., Fujimoto, M., Shagan, D.S., Seltzer, J.C., 2008. Symptoms versus neurocognition as predictors of change in life skills in schizophrenia after outpatient rehabilitation. Schizophr. Res. 102, 303-311. http://dx.doi.org/10.1016/ j.schres.2008.03.023.

Lal, S., Dell'Elce, J., Tucci, N., Fuhrer, R., Tamblyn, R., Malla, A., 2015. Preferences of young adults with first-episode psychosis for receiving specialized mental health services using technology: a survey study. JMIR Ment. Health 2, e18. http://dx.doi. org/10.2196/mental.4400.

Lindenmayer, J.-P., McGurk, S.R., Khan, A., Kaushik, S., Thanju, A., Hoffman, L., Valdez, G., Wance, D., Herrmann, E., 2013. Improving social cognition in schizophrenia: a pilot intervention combining computerized social cognition training with cognitive remediation. Schizophr. Bull. 39, 507-517. http://dx.doi.org/10.1093/schbul/ sbs120.

Lumsden, J., Edwards, E.A., Lawrence, N.S., Coyle, D., Munafo, M.R., 2016. Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 4, e11. http://dx.doi.org/10.2196/games.5888.

McGurk, S.R., Twamley, E.W., Sitzer, D.I., McHugo, G.J., Mueser, K.T., 2007. A metaanalysis of cognitive remediation in schizophrenia. Am. J. Psychiatry 164, 1791-1802. http://dx.doi.org/10.1176/appi.ajp.2007.07060906.

Medalia, A., Saperstein, A., 2011. The role of motivation for treatment success. Schizophr. Bull. 37 (Suppl. 2), S122-128. http://dx.doi.org/10.1093/schbul/sbr063.

Nahum, M., Fisher, M., Loewy, R., Poelke, G., Ventura, J., Nuechterlein, K.H., Hooker, C.I., Green, M.F., Merzenich, M.M., Vinogradov, S., 2014. A novel, online social cognitive training program for young adults with schizophrenia: a pilot study. Schizophr. Res. Cogn. 1, e11-e19. http://dx.doi.org/10.1016/j.scog.2014.01.003.

Nuechterlein, K.H., Green, M.F., Kern, R.S., Baade, L.E., Barch, D.M., Cohen, J.D., Essock,

S., Fenton, W.S., Frese, F.J., Gold, J.M., Goldberg, T., Heaton, R.K., Keefe, R.S., Kraemer, H., Mesholam-Gately, R., Seidman, L.J., Stover, E., Weinberger, D.R., Young, A.S., Zalcman, S., Marder, S.R., 2008. The MATRICS consensus cognitive battery, part 1: test selection, reliability, and validity. Am. J. Psychiatry 165, 203-213. http://dx.doi.org/10.1176/appi.ajp.2007.07010042.

Patterson, T.L., Goldman, S., McKibbin, C.L., Hughs, T., Jeste, D.V., 2001. UCSD performance-based skills assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr. Bull. 27, 235-245.

Subramaniam, K., Luks, T.L., Fisher, M., Simpson, G.V., Nagarajan, S., Vinogradov, S., 2012. Computerized cognitive training restores neural activity within the reality monitoring network in schizophrenia. Neuron 73, 842-853. http://dx.doi.org/10. 1016/j.neuron.2011.12.024.

Subramaniam, K., Luks, T.L., Garrett, C., Chung, C., Fisher, M., Nagarajan, S., Vinogradov,

S., 2014. Intensive cognitive training in schizophrenia enhances working memory and associated prefrontal cortical efficiency in a manner that drives long-term functional gains. NeuroImage 99, 281-292. http://dx.doi.org/10.1016/j. neuroimage.2014.05.057.

Ventura, J., Wilson, S.A., Wood, R.C., Hellemann, G.S., 2013. Cognitive training at home in schizophrenia is feasible. Schizophr. Res. 143, 397-398. http://dx.doi.org/10. 1016/j.schres.2012.11.033.

Original Paper

ReMindCare App for Early Psychosis: Pragmatic Real World Intervention and Usability Study

Lucia Bonet1,2, MSci, PhD; John Torous3, MD; David Arce4, MSci; Ignacio Blanquer4, Prof Dr; Julio Sanjuan1,2,5, MD

department of Mental Health, Sanitary Research Institute of Valencia, University Clinic Hospital of Valencia, Valencia, Spain

2Faculty of Medicine and Odontology, University of Valencia, Valencia, Spain

3Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States 4Institute of Instrumentation for Molecular Imaging, Joint Centre of the Spanish National Research Council and Universitat Politecnica de Valencia, Valencia, Spain

5Centre of Biomedical Investigation in Mental Health, Spanish Government Carlos III Health Institute, Madrid, Spain

Corresponding Author:

Julio Sanjuan, MD

Department of Mental Health

Sanitary Research Institute of Valencia

University Clinic Hospital of Valencia

Menendez y Pelayo St, 4,

Valencia

Spain

Phone: 34 961 97 35 17

Email: iulio.sanjuan@uv.es

Abstract

Background: eHealth interventions are widely used in clinical trials and increasingly in care settings as well; however, their efficacy in real-world contexts remains unknown. ReMindCare is a smartphone app that has been systematically implemented in a first episode of psychosis program (FEPP) for patients with early psychosis since 2018.

Objective: The objective of this study was to assess the efficacy of ReMindCare after 19 months of use in the clinic and varying use by individual patients.

Methods: The integration of the ReMindCare app into the FEPP started in October 2018. Patients with early psychosis self-selected to the app (ReMindCare group) or treatment as usual (TAU group). The outcome variables considered were adherence to the intervention and number of relapses, hospital admissions, and visits to urgent care units. Data from 90 patients with early psychosis were analyzed: 59 in the ReMindCare group and 31 in the TAU group. The mean age of the sample was 32.8 (SD 9.4) years, 73% (66/90) were males, 91% (83/90) were White, and 81% (74/90) were single.

Results: Significant differences between the ReMindCare and TAU groups were found in the number of relapses, hospitalizations, and visits to urgent care units, with each showing benefits for the app. Only 20% (12/59) of patients from the ReMindCare group had a relapse, while 58% (18/31) of the TAU patients had one or more relapses (x2=13.7, P=.001). Moreover, ReMindCare patients had fewer visits to urgent care units (X2=7.4, P=.006) and fewer hospitalizations than TAU patients (x2=4.6, P=.03). The mean of days using the app was 352.2 (SD 191.2; min/max: 18-594), and the mean of engagement was 84.5 (SD 16.04).

Conclusions: To our knowledge, this is the first eHealth intervention that has preliminarily proven its benefits in the real-world treatment of patients with early psychosis.

International Registered Report Identifier (IRRID): RR2-10.1111/eip.12960

(JMIR Mhealth Uhealth 2020;8(11):e22997) doi: 10.2196/22997

KEYWORDS

app; clinical practice; mental health; psychosis; real-world intervention; telemedicine

Introduction

High interest in eHealth services and now digital and mobile health has been noted in many recent studies among patients with psychotic disorder diagnoses [1,2]. With COVID-19, this interest in digital health has surged, and the need to expand access to care through smartphones has become patent. Smartphone apps have been proposed as tools to mitigate social isolation, lack of access to care, and other triggers caused by the pandemic [3-5]. Researchers have already demonstrated that access to and use of technology among people with psychosis is nearly equivalent to that in the general population [6-8], but less is known about the actual efficacy of apps in care.

Apps have already seen growth in care for patients with early course psychosis. Many studies are using real-time ecological momentary assessment (EMA) surveys to monitor symptoms and experiences and identify early indicators of relapse [9]. Beyond relapse prediction, these EMA data can offer novel information on the longitudinal health status of patients, which could improve treatment and shared decision making between patient and physician [10]. Finally, eHealth services may be a major resource to enhance the benefits of the first episode of psychosis programs (FEPPs) for early psychosis, which can foster recovery [11] and reduce the risk of hospitalization and relapse [12,13].

Specific apps targeting schizophrenia have already been created and offer promising results. Examples of these innovative interventions are the Actissist [14] and the ExPRESS [15] interventions, which demonstrated potential in improving the quality of treatment of patients with early psychosis. Another example is the CrossCheck app [16], which demonstrated potential for identifying and dismantling dysfunctional beliefs that contribute to maintenance and distress associated with psychotic symptoms. Despite the widespread use of these eHealth interventions and high rates of efficacy reported in clinical trials, the efficiency and actual efficacy of these interventions in real-world clinical practice remains unknown [17].

One reason for the lack of initial success of health apps in clinical settings is lack of engagement. Often engagement in academic studies does not translate into real-world use [18,19]. Indeed, some studies found a negative correlation between the time spent using eHealth apps and the engagement of patients [20,21]. In addition, many clinicians expressed their concern that if these systems integrate seamlessly with clinical workflow, they will result in an increase in the clinicians' workload [22,23], which might affect their engagement with the app.

Other concerns have also limited efforts to integrate these apps into care settings. In our previous study [8], we found that 20% to 23% of patients felt anxious, suspicious, or paranoid concerning the internet, and almost 25% of patients perceived that use of the internet was directly related to one of their relapses. In addition, some studies indicated that excessive eHealth communications could be regarded as intrusive or irritating [24,25] or could increase worries about illness [25]. These potential harms of eHealth interventions must also be taken into consideration.

Considering these factors, it is clear that eHealth interventions shown to be feasible must now be assessed for effectiveness, efficacy, and efficiency [26] in real-world settings. With this objective in mind and to improve the daily treatment of patients with psychosis, we designed the ReMindCare app. The protocol followed for the design process and implementation of the app is published elsewhere [27]. In this protocol, we introduced ReMindCare as a smartphone app plus a clinician dashboard, developed to be implemented in a FEPP for patients with early psychosis.

To the best of our knowledge, ReMindCare is the first eHealth intervention for patients with early psychosis that has been systematically integrated into daily clinical practice, finally filling the gap between research and clinical practice [2,17].

The aim of this study was to assess the efficacy and clinical outcomes of the use of the app after 19 months in terms of adherence to ReMindCare, relapse prevention, hospital admissions, and visits to urgent care units compared with treatment as usual (TAU) without the app.

Methods

Study Setting

The app was systematically integrated into the daily clinical workflow in a FEPP at the University Clinic Hospital of Valencia, Spain. This FEPP started in 2010 with the objective of improving early detection, evaluation, and personalization of treatment. It covers a total of 330,000 inhabitants included in Area 5 of Valencia city. The incidence of novel psychotic disorders in this area has gradually increased during the 10 years since the program started. Currently, the FEPP in the clinic hospital has a mean of 30 to 35 new patients with psychosis per year.

The implementation of the ReMindCare app into the FEPP and into clinical practice started in October 2018 and is still in use today. In this study, we present the results from the first 19 months of use of the app.

Neither patients nor physicians received any remuneration or compensation for participating in the program or using the app. The use of the app was offered as an extra free service to the patients in the program.

Participants

Recruitment and Enrollment

The patient's psychiatrist of reference offered the use of the ReMindCare app to every outpatient from the FEPP who met the criteria for inclusion. Once patients enrolled in the study, they were encouraged to use the app as long as they remained in the program (maximum period of 5 years). To use the app, all patients signed an informed consent form and completed baseline assessments.

Eligibility Criteria

To be considered for this intervention, patients met the following criteria: (1) diagnosis of psychotic disorder following DSM-5 (Diagnostic and Statistical Manual of Mental Disorders, 5th Edition) criteria, interview conducted by a licensed clinician, (2) aged between 17 and 65 years, (3) smartphone ownership with an internet connection that allows for the proper installation and functioning of the app, and (4) less than 5 years of illness duration. However, it must be stated that some patients remained in the program for more than 5 years. These patients remained in the FEPP to prevent potential relapses, as they experienced severe fluctuations in their symptoms.

Criteria for exclusion were (1) lack of ability to use and master a mobile device and the internet, (2) refusal to sign an informed consent form, and (3) level of Spanish or English not fluent enough to maintain a conversation or understand the app questionnaires.

Intervention

ReMindCare App

ReMindCare is a free and user-friendly app that conducts daily evaluations of the health status of patients with early psychosis by offering quick questionnaires (Figure 1).

Two types of questionnaires were included:


Figure 2. Screenshot of the ReMindCare daily questionnaire.

ReMindCare

Daily Evaluation

Are you feeling depressed? 9

* Not at all Slightly      Moderately      Very      Extremely

Are you feeling nervous? 9

Not at all • Slightly      Moderately      Very      Extremely

I hardly got a couple of hours of sleep last night

Are you getting angry easily? 9

Not at all Slightly C Moderately Very Extremely

Do you want to leave a comment? Maximum 140 characters

Next                1/5

In addition, the app offered preset alerts in case of low engagement or abrupt changes in survey responses. Low engagement alerts were set off if patients did not respond to the surveys for 7 days or more, while abrupt changes were considered when there was a difference of 2 points (Likert scale 1 to 5) or more between each question in the last 2 surveys answered. These alerts notified physicians by email and were also displayed in the profile of the patient on the app's website portal.

All data captured by the app were accessible for physicians on a password-protected dashboard. Moreover, physicians could download a summary pdf of these data from the dashboard and attach it to the electronic clinical record of the patient in the hospital database.

The app is available in 3 languages (Spanish, English, and Catalan), although we are open to developing new language versions of the app. Our aim is to extend the use of the app to other countries, and adaptation of the app to different languages would be necessary to ensure patient engagement. Further information about the design process of the app and its characteristics can be found in the ReMindCare app study protocol [27].

Patients who used the app (ReMindCare group) did not experience any changes in their usual clinical appointments.

Treatment as Usual

The TAU group comprised patients who met the criteria but rejected using the app. In this group of patients, 42% (13/31) were patients with low adherence to treatment, 26% (8/31) did not perceive any benefit from using the app, and 26% (8/31) were suspicious about technology and their privacy. Additionally, 6% (2/31) were included in this group because they only used the app for 2 days. These patients continued with their usual psychiatric treatment at the FEPP and were not adversely affected by their rejection of participation.

Procedure

Once patients enrolled in the FEPP, after an interview with their psychiatrist of reference, they were asked to complete some baseline assessments. Subsequently, they were offered the use of the ReMindCare app. The ReMindCare app was described as an extra tool developed by the FEPP that could help them manage their symptoms and help clinicians better understand their illness evolution. The main characteristics of the app were listed. After receiving this information, patients decided whether they were willing to use the app. If they were not interested, they were placed in the TAU group. If patients were interested, they were informed in more detail by an expert clinician about the installation process, characteristics of the app, and ethics and data privacy information.

Patients could use the ReMindCare app to contact their psychiatrist of reference directly in case of symptoms worsening by using the urgent consultation request tab on the home screen of the app. If they clicked the urgent consultation request, their clinician would contact them by phone within 48 hours (patients who did not use the app could call the department of psychiatry at the hospital and be referred to their psychiatrist or attend an urgent care unit). In addition, clinicians contacted patients by phone in response to preset alarms. As a result of these phone calls and the information that patients provided to the clinician, urgent care visits could be scheduled if necessary. With these services, we aimed to improve the detection of early psychotic symptoms and reduce the visits to urgent care units at the hospital, as these prodromal symptoms will be primarily treated by a phone call or in the outpatient services. If patients did not make an urgent consultation request and no preset alarms were set off, they continued with their scheduled clinical appointments.

Furthermore, the use of the ReMindCare app changed the dynamics of the clinical appointment at the outpatient services. Once patients arrived at the clinical appointment, physicians accessed their profile on the ReMindCare's physician dashboard and used the information provided for patients to guide them through the interview. Clinicians used shared decision making with patients and discussed their responses.

Data Collection and Measures

Baseline

After patients were enrolled in the FEPP, the following data were collected:

Outcome Measures

Data Analysis

Data were analyzed with the statistical program SPSS Statistics version 22 (IBM Corp). The cohort was divided into two groups: ReMindCare group patients agreed to use the app and used it for at least 1 month; the TAU group patients did not use the app or used it for less than 1 month. To consider that patients in the ReMindCare group had a relapse while using the app, patients had to be actively using the app. Relapses of patients who did not use the app for more than 2 months were not considered as relapses while using the app. Descriptive statistics (mean, standard deviation, frequency, and percentage) were determined, and chi-square test analysis was performed to compare the differences between the ReMindCare group and the TAU group.

Ethics, Data Privacy, and Participant Safety

The ReMindCare app project received approval from the research ethics committee of the faculty of medicine at the University of Valencia and from the research ethics committee of the Sanitary Research Institute of the University Clinic Hospital of Valencia, Spain.

To protect the data sent by patients, communications to the platform were encrypted with a transport layer security certificate from the Generalitat Valenciana and were sent through the https protocol. The hospital infrastructure is protected through a reverse proxy, which enhances security by establishing a single access point to it and hiding all inner infrastructures. Moreover, the integration of the app into the hospital systems was subjected to Organic Law 3/2018: protection of personal data and digital rights guarantee, December 5th, the Spanish organic law adaptation of the General Data Protection Regulation.

Results

Data from 90 patients were analyzed: 59 used or are using the app (ReMindCare group) and 31 did not agree to use the app (TAU group). Characteristics of both groups are displayed in Tables 1 and 2.

JMIR MHEALTH AND UHEALTH

Table 1. Sociodemographic data.

Bonet et al

Characteristic

Total

RCa group

TAUb

X2 (P value)

Age in years, mean (SD)

32.8 (9.4)

32.1 (1.2)

34.3 (1.7)

1.5 (.57)

24 and younger, n (%)

19 (21)

12 (20)

7 (23)

c

25-44, n (%)

58 (64)

40 (68)

18 (58)

45 and older, n (%)

13 (14)

7 (12)

6 (19)

Gender (male), n (%)

66 (73)

40 (68)

25(81)

1.7 (.19)

Native country (Spain), n (%)

79 (87)

48 (81)

30 (97)

4.2 (.04)

Race (White), n (%)

83 (91)

51 (86)

31(100)

4.6 (.33)

Marital status, n (%)

5.2 (.16)

Single

74 (81)

50 (85)

23 (74)

Married

11 (12)

5 (9)

6 (19)

Other

85 (7)

4 (7)

2 (7)

Educational level, n (%)

5.9 (.05)

Primary

2 (2)

0 (0)

2 (7)

Secondary

45 (50)

27 (46)

18 (58)

College or higher

43 (48)

32 (54)

11 (36)

Employment status, n (%)

5.6 (.24)

Employed

29 (32)

16 (27)

13 (42)

Student

21 (23)

16 (27)

4 (13)

Not employed

38 (42)

25 (42)

13 (42)

Unable to work

3 (3)

2 (3)

1 (3)

Cohabitation, n (%)

2.3 (.51)

Alone

6 (7)

3 (5)

3 (10)

Family_birth

60 (66)

39 (66)

20 (65)

Family_own

11 (12)

6 (10)

5 (16)

Other

14 (15)

11 (19)

3 (10)

aRC: ReMindCare. bTAU: treatment as usual. cnot applicable.

Table 2. Baseline clinical information.

Characteristics

Total

RCa group

TAUb

X2 (P value)

Injectable medication, n (%)

18 (20)

8 (14)

10 (32)

4.4 (.03)

Length of illness in years, mean (SD)

10.5 (2.8)

3.9 (0.4)

5.7 (0.5)

12.3 (.002)

0-1, n (%)

13 (14)

13 (22)

0 (0)

___c

2-5, n (%)

43 (48)

30 (51)

13 (42)

More than 6, n (%)

34 (38)

16 (27)

18 (58)

Associated illnesses, n (%)

29 (32)

18 (31)

11 (36)

0.2 (.63)

Suicidal attempts, n (%)

16 (18)

12 (22)

3 (10)

2.1 (.15)

CGI-SId, mean (SD)

4.2(0.9)

4.1 (0.1)

4.4 (0.1)

2.7 (.26)

Mild (1-3), n (%)

13 (16)

10 (19)

3 (11)

Moderate (4-5), n (%)

66 (83)

42 (81)

24 (86)

Severe (>5), n (%)

1 (1)

0 (0)

1 (4)

GAFe, mean (SD)

60.7 (10.9)

61.3 (1.7)

59.8 (1.7)

1.3 (.52)

Mild (71-100), n (%)

8 (10)

4 (8)

4 (14)

Moderate (51-70), n (%)

51 (65)

35 (69)

16 (57)

Severe (<50), n (%)

20 (25)

12 (24)

8 (29)

PANSSf, mean (SD)

65.9 (18.8)

64.5 (2.2)

68.7 (4.6)

52.1 (.28)

Positive

18.4 (6.5)

18.7 (5.8)

18.7 (6.8)

23.9 (.58)

Negative

18.9 (6.9)

15.4 (5.1)

17.9 (9.3)

28.2 (.17)

N5. Difficulty in abstract thinking

2.3 (1.3)

2.0 (0.2)

2.8 (1.5)

12.8 (.03)

N6. Lack of spontaneity and flow conversation

1.7 (1.3)

1.6 (1.1)

1.9 (1.7)

12.9 (.02)

General

32.3 (8.2)

66.1 (14.7)

70.5 (22.2)

32.2 (.41)

G5. Mannerism and posturing

1.1 (0.7)

1.1 (0.4)

1.3 (0.7)

9.9 (.01)

PASg, mean (SD)

10.5 (2.8)

10.7 (0.5)

10.14 (0.6)

9.1 (.70)

Relapses_Baseline, n (%)

4.3 (.12)

0

53 (59)

38 (64)

15 (48)

1

21 (23)

14 (24)

7 (23)

>2

16 (18)

7 (12)

9 (29)

UCUh visits_Baseline, n (%)

0.9 (.61)

0

26 (29)

19 (32)

7 (23)

1

36 (40)

23 (39)

13 (42)

>2

28 (31)

17 (29)

11 (36)

Hospitalizations_Baseline, n (%)

4.6 (.10)

0

19 (21)

16 (27)

3 (10)

1

50 (56)

32 (54)

18 (58)

>2

21 (23)

11 (19)

10 (32)

aRC: ReMindCare.

bTAU: treatment as usual.

cnot applicable.

dCGI-SI: Clinical Global Impression Severity of Illness scale eGAF: Global Assessment of Functioning.

f

PANSS: Positive and Negative Syndrome Scale. gPAS: Premorbid Adjustment Scale.

https://mhealth.jmir.org/2020/11/e22997

XSL-FO

Table 3. Use of ReMindCare.

Characteristic

RCa group (n=59)

Min-max

Days using app, mean (SD)

352.2 (191.2)

18-594

Months using app, mean (SD)

11.6 (6.5)

0-19

Engagement, mean (SD)

84.5 (16.0)

42-100

85%-100%, n (%)

36(61)

__b

UCUc, n (%)

18 (31)

Relapses using app, n (%)

12 (20)

_

Relapses related to app, n (%)

2 (8)

_

Status of use after 19 months, n (%)

Patients using app

37 (63)

_

Patients not using app

22 (37)

_

Discharged from FEPPd

7 (32)

_

Dropouts

15 (68)

_

aRC: ReMindCare.

bnot applicable.

cUCU: urgent care units.

dFEPP: first episode of psychosis program.

Table 4. Clinical outcomes after 19 months of the ReMindCare intervention.

Characteristic

Total, n (%)

RCa group, n (%)

TAUb, n (%)

X2 (P value)

Relapses

___c

_

_

13.7 (.001)

0

60 (67)

47 (80)

13 (42)

_

1

29 (32)

12 (20)

17 (55)

_

>2

1 (1)

0 (0)

1 (3)

_

UCUd visits

20 (22)

8 (14)

12 (39)

7.4 (.006)

Hospitalizations

9 (10)

3 (5)

6 (19)

4.6 (.03)

aRC: ReMindCare. bTAU: treatment as usual. cnot applicable.

dUCU: urgent care units.

hUCU: urgent care units.

Sociodemographic Analysis

The mean age of the sample was 32.8 (SD 9.4) years, 73% (66/90) were males, 91% (83/90) were White, and 81% (74/90) were single. No significant differences were found between the ReMindCare and TAU groups in any of the sociodemographic information analyzed except for the native country. We found that nearly every immigrant considered for inclusion agreed to use the app (ReMindCare group 19% [10/11], TAU group 3% [1/11]; X2=4.2, P=.04). Further information regarding sociodemographic analysis of the data is displayed in Table 1.

Baseline Clinical Analysis

Significant differences were found between the ReMindCare group and TAU group in some clinical factors. With regard to injectable medication, 32% (10/31) of TAU patients were taking injectable medication, while only 14% (8/59) of the ReMindCare took it (x2=4.4, P=.04). Every new patient in the FEPP (length of illness: 0-1 year) agreed to use the app (13/90, 22%), and 58% (18/31) of the TAU group had their illness for 6 or more years (%2=12.3, P=.002). Moreover, the TAU patients showed higher scores on the PANSS N5 and N6 negative subscales and G5 in the general subscales (x2=12.8, P=.03; x2=12.9, P=.02; X2=9.9, P=.01, respectively).

Considering medication, 20% (18/90) of patients were taking injectable medications, 32% (29/90) of the patients suffered from another illness, and 18% (17/90) had a prior suicidal attempt. The mean of the CGI-SI was 4.2 (SD 0.9), the GAF mean=60.7 (SD 10.9), PANSS mean 65.9 (SD 18.8), and PAS


mean 10.5 (SD 2.8). Finally, 12% (11/90) of patients were discharged from the FEPP. No significant differences were found between the groups in any of these factors. Moreover, no significant differences were found between the ReMindCare group and TAU group in terms of the number of relapses (X2=4.3, P=.12), visits to urgent care units (x2=0.9, P=.61), or the number of hospitalizations (x2=4.6, P=.10) at baseline. Further clinical information is available in Table 2.

ReMindCare Outcomes

The mean of days using the app was 352.2 (SD 191.2), which corresponds to 11.6 months. The mean of compliance was 84.5 (16.04), and 61.1% of the ReMindCare group had a compliance rate between 85% and 100%.

Of the 59 ReMindCare patients, 31% (18/59) requested an urgent consultation, 20% (12/59) had a relapse while using the app, and 8% (2/59) developed a delusion involving the app and the research group.

After 19 months of intervention, 63% (37/59) of patients continued using the app, while 12% (7/59) stopped using the app because they were discharged from the FEPP and 25% (15/59) opted to stop using ReMindCare. Reasons for discontinuation: 33% (5/15) of patients felt suspicious about technology (among these patients, 4 had a relapse while using the app); 40% (6/15) perceived the app as boring and did not perceive any benefit; and 27% (4/15) of patients left treatment and did not continue in the program. This information is shown in Table 3.


With regard to the clinical outcomes, after 19 months of ReMindCare’s integration into the clinical workflow, only 20% (12/59) of patients from the ReMindCare group had a relapse, while 58% (18/31) of TAU patients had one or more relapses


(X2=13.7, P=.001). Moreover, ReMindCare patients had fewer visits to urgent care units (X2=7.4, P=.006) and fewer hospitalizations than TAU patients (%2=4.6, P=.03). Information regarding these clinical outcomes is displayed in Table 4.


Discussion


Principal Findings

The results obtained from these analyses of the first 19 months of ReMindCare use highlight the potential benefits of this eHealth intervention for patients with early psychosis. Patients who used the app not only had fewer relapses than the TAU group, but they also had fewer visits to the urgent care unit and fewer hospitalizations.

Results related to the efficacy of the app are in line with previous results obtained in clinical trials [14-16]. However, as far as we know, this is the first study to identify the benefits of the use of an app as a tool systematically integrated into daily clinical practice in a FEPP.

With regard to the feasibility of the app, no significant differences were found between the ReMindCare group and the TAU group in terms of sociodemographic characteristics except for native country. The feasibility of this intervention aligns with the results obtained in our previous study [8], where we found no differences in terms of sociodemographic characteristics and interest in using eHealth interventions.

With regard to the clinical characteristics of the samples and their impact on the effect of ReMindCare, there were some differences between groups. We found that patients who did not use the app were more likely to be taking injectable medication, have a longer history of illness, and have higher scores on the PANSS N5 and N6 negative subscales and G5 in the general subscales. These results might suggest that the use of ReMindCare was not indicated for chronic patients. However, we did not find differences in other clinical scales such as the CGI-SI, GAF, and PAS scales or even on the PANSS total scale. More importantly, we did not find any differences between groups in terms of baseline relapses, hospitalizations, or visits to urgent care units.


These results are in line with the ones we obtained in our previous study [8], where we found that interest in using eHealth apps was equivalent between chronic and early psychosis patients. In this regard, we suggest that differences obtained in terms of the clinical characteristics of the patients could be more related to the history of treatment than to clinical characteristics. As we found, every new patient who joined the FEPP (length of illness less than 1 year) was interested in using the app (22% of users), while patients who had a longer history of treatment (length of illness more than 6 years) were more likely to reject its use (58% of TAU group). This could highlight the relevance of introducing these new technologies at the very beginning of treatment so early psychosis patients consider these apps to be just another tool included in their daily clinical treatment and not an extra service, especially since our results suggested that use of the app had a significant impact in improving the course of the illness.

Finally, with regard to compliance and engagement with the app, we found that 61% of patients had compliance rates between 85% to 100%. Rates of engagement were also high, as 63% of patients still use the app after almost 1 year. These results of compliance and long-term engagement are contrary to previous studies [20,21] and suggest that the use of an app in a long-term approach is feasible and beneficial.

However, we would like to highlight that 20% of patients had a relapse while using the app and 8% developed a delusion involving the use of the app and the research group. These negative results should be cautiously considered.

Technology could be a major resource to improve the quality of treatments, but as we found in a previous study [8], it can also play an important role as a trigger for psychotic symptoms. In this regard, in a 3-case study in 2011 conducted by Nitzan et al [32], they stated that the use of the internet and computers might contribute to a gradual break with reality and development of psychotic symptoms. They suggested that given that patients


with psychotic diagnoses have greater difficulties in filtering and understanding signals and symbols, they are also more likely to misinterpret digital messages. However, no specific studies regarding the potential harms of the use of new technologies have been undertaken until the present.

In our study, we found that the ReMindCare app was related to beneficial clinical effects for the vast majority of patients who used it. However, despite the general positive effects found in this study, there are still some barriers and negative effects that must be taken into consideration. The main barrier found in our study relates to the 34% of the approached patients who did not want to use the app and who also tended to be the more chronic patients. Moreover, the main negative effect we found related to the 8% of patients who developed a delusion involving the app. As a result, we would like to point out that this app is not a panacea to prevent relapses. However, it is clear that the app positively affected the course of the illness, as only 5% of those who relapsed required hospitalization compared with 19% of patients who relapsed in the TAU group.

Limitations and Strengths

There were some limitations that must be taken into consideration. First, not every outpatient from the FEPP was eligible for inclusion, as some patients did not have their own smartphone with an internet connection or did not have the ability to use the app or understand it due to language barriers. Developing strategies to prevent digital exclusion should be a priority to ensure that every patient could benefit from these technologies [33]. Second, as a real-world study, this study was not randomized. Despite the groups not differing in the vast majority of clinical or demographic characteristics, there were some factors such as personality that could influence our results.

The main strength of our study was the fact that ReMindCare is the first app that has been systematically integrated into the clinical FEPP workflow. To our knowledge, there are no previous studies that used an app as a tool to improve the daily treatment of patients with early psychosis. All the studies we found were conducted in academic research settings that did not emulate real-world environments [17,34].

Another strength is in regard to the development of the ReMindCare app. First, it was based on two previous studies [2,8] and co-designed with patients [27]. Second, we conducted a pilot study and focus groups to ensure the involvement of both patients and care providers [27] in the design and improvement process of the app.

Finally, we would like to highlight the long-term approach of this intervention. As stated before, ReMindCare is now


Bonet et al integrated into clinical practice and it was used for 19 months. These results align with previous studies [16] that found that people with psychosis have the abilities and interest required to engage in long-term eHealth interventions.

Implications for the Future

As a result of these analyses, we highlighted the benefits that the use of ReMindCare app produced on early psychosis patients in a FEPP. Our aim is to continue improving the app in response to the needs and suggestions provided by patients and clinicians. As Ross et al [22] claimed in their meta-review, in order to ensure the use of these eHealth technologies over time, there are three challenges that should be overcome. First, the apps must be able to adapt to the characteristics of the environment and patients. Second, the apps should be easy to use. Third, the apps should be integrated into clinical practice, adjusting the characteristics of the app in order to ensure it is user-friendly and efficient for patients and clinicians. It is our aim to address these issues to maintain the positive results obtained in this study.

However, we would like to point out a major issue that must guide future eHealth interventions. As stated before, 8% of patients developed a delusion related to the use of the app, 25% of patients deliberately stopped using the app, and 34% of patients approached did not want to use the app in the first place. These results suggest that there are still significant numbers of patients not willing to use eHealth interventions, and there are some patients who could be adversely affected by the use of these technologies. Studying the characteristics of these patients should guide future research in order to ensure that the use of digital technologies only provides benefits to the patients [8].

Finally, we would like to underline that given the exceptional situation that the world is facing at the moment with COVID-19 and in order to address the requirements of interventions that could improve the telematic treatment of patients and prevention of hospital collapses [4,35], ReMindCare could be used as an effective and efficient tool. Since quarantining in Spain began March 13, 2020, patients have not been permitted to come in person to their clinical appointments and have received their clinical evaluations by phone. Since that moment, the use of ReMindCare has been extremely useful to improve the evaluation and adherence of early psychosis patients. However, future analysis will be conducted in regard to this aspect.

As the conclusion of this study, we would like to point out that, to the best of our knowledge, ReMindCare is not only the first app to be integrated into the clinical practice, it is the first eHealth intervention with evidence that it improves the outcomes of early psychosis patients in a real-world care setting.


Acknowledgments

This study was supported by the Sanitary Research Institute of the University Clinic Hospital and the Mental Health Networking Biomedical Centre. It was also supported by the Generalitat Valenciana and the Program for Scientific Research, Technological Development, and Innovation in the Generalitat Valenciana of the European Union (2017/9830). It was also supported by grants from the Generalitat Valenciana (PROMETEO/2016/082, PROMETEO/2020/024), Carlos III Health Institute (PI13/00447; PI17/00402), and European Union through the European Regional Development Fund.

Authors' Contributions

LB and JS recruited and evaluated the patients. JS treated the patients, and LB supervised the app performance and the patients’ responses and alarms. DA and IB developed the app and supervised its functioning, and they also drafted and discussed the paper. LB wrote the paper and analyzed the data. JS designed and supervised the project. JS and JT reviewed the data and supervised the final version of the manuscript.

Conflicts of Interest

None declared.

References

Abbreviations

CGI-SI: Clinical Global Impression Scale-Severity Illness

DSM-5: Diagnostic and Statistical Manual of Mental Disorders, 5th Edition

EMA: ecological momentary assessment

FEPP: first episode of psychosis program

GAF: Global Assessment of Functioning

PANSS: Positive and Negative Syndrome Scale

PAS: Premorbid Adjustment Scale

TAU: treatment as usual

Edited by L Buis; submitted 29.07.20; peer-reviewed by J Li, L Levin, MDG Pimentel; comments to author 15.09.20; revised version received24.09.20; accepted 16.10.20; published06.11.20

Please cite as:

Bonet L, Torous J, Arce D, Blanquer I, Sanjuan J

ReMindCare App for Early Psychosis: Pragmatic Real World Intervention and Usability Study

JMIR Mhealth Uhealth 2020;8(11):e22997

URL: https://mhealth.jmir.org/2020/11/e22997

doi: 10.2196/22997

PMID: 33155986

©Lucia Bonet, John Torous, David Arce, Ignacio Blanquer, Julio Sanjuan. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 06.11.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.

Actissist: Proof-of-Concept Trial of a Theory-Driven Digital Intervention for Psychosis

Sandra Bucci*,1, Christine Barrowclough1, John Ainsworth2,3, Matthew Machin2,3, Rohan Morris1, Katherine Berry1, Richard Emsley4, Shon Lewis1, Dawn Edge1, Iain Buchan5, and Gillian Haddock1

1Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK; 2Division of Informatics Imaging and Data Sciences, University of Manchester, Manchester, UK; 3Health eResearch Centre, Farr Institute for Health Informatics Research, University of Manchester, Manchester, UK; 4Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, UK; 5Microsoft Research, Cambridge, UK

*To whom correspondence should be addressed; Division of Psychology and Mental Health, University of Manchester, 2nd Floor, Zochonis Building, Brunswick Street, Manchester M13 9PL, UK; tel: +44-161-306-0422, fax: +44-161-306-0402, e-mail: sandra.bucci@ manchester.ac.uk

Background: Timely access to intervention for psychosis is crucial yet problematic. As such, health care providers are forming digital strategies for addressing mental health challenges. A theory-driven digital intervention that monitors distressing experiences and provides real-time active management strategies could improve the speed and quality of recovery in psychosis, over and above conventional treatments. This study assesses the feasibility and acceptability of Actissist, a digital health intervention grounded in the cognitive model of psychosis that targets key early psychosis domains. Methods: A proof-of-concept, single, blind, randomized controlled trial of Actissist, compared to a symptom-monitoring control. Thirty-six early psychosis patients were randomized on a 2:1 ratio to each arm of the trial. Actissist was delivered via a smartphone app over 12-weeks; clinical and functional assessment time-points were baseline, post-treatment and 22-weeks. Assessors’ blind to treatment condition conducted the assessments. Acceptability was examined using qualitative methods. Results: Actissist was feasible (75% participants used Actissist at least once/day; uptake was high, 97% participants remained in the trial; high follow-up rates), acceptable (90% participants recommend Actissist), and safe (0 serious adverse events), with high levels of user satisfaction. Treatment effects were large on negative symptoms, general psychotic symptoms and mood. The addition of Actissist conferred benefit at post-treatment assessment over routine symptom-monitoring and treatment as usual. Conclusions: This is the first controlled proof-of-concept trial of a theory-driven digital health intervention for early psychosis. Actissist is feasible and acceptable to early psychosis patients, with a strong signal for treatment efficacy. Trial Registration: ISRCTN: 34966555.

Key words: psychosis/relapse/mHealth/digital intervention/randomized controlled trial/early psychosis.

Psychosis onset typically occurs in early adulthood, a critical period for psychosocial development. Despite initial response to intervention, the early course of psychosis is characterized by repeated relapse,1 compromising functional and social development,2 service engagement,2 and the resilience of carers and services.3 Meta-analyses have shown that discontinuation of antipsychotic medication, substance misuse, family criticism, poorer premorbid functioning, and social isolation are firmly associated with relapse following a first episode of psychosis (FEP1). Early intervention for psychosis services (EIS) exist worldwide and aim to provide both pharmacological and psychosocial interventions. Despite mandates published to address the treatment gap,4 timely access to these services is problematic.5

Health care providers worldwide are forming digital strategies for addressing mental health challenges, and self-management in long-term conditions is now a cornerstone of many national health policies.6 Smartphones are commonplace technology that can deliver unconstrained, real-time packages of care, extending the reach of health care delivery. Smartphone-extended care could drive improvements in quality, efficiency, cost, and access to treatment, while enhancing patient experience by: providing more choices over how health care is delivered; facilitating self-management; and assisting clinicians to gain a richer understanding of an individual’s day-to-day experiences by receiving real-time data that can be used to deliver ecologically valid treatment. Smartphone ownership rates in psychosis are comparable to the general population7 and use of smartphones for health care appears acceptable to people with severe mental health problems.8 Large-scale meta-analyses of randomized controlled trials (RCTs) have demonstrated that digital health interventions (DHIs) can provide effective treatment for mental health problems such as depression9 and anxiety.10 While a number of smartphone-delivered open trials have shown promising effects in reducing hospital admissions, improving positive psychotic symptoms, socialization, social connectedness, depression, and medication adherence (see review1), a user-informed, theory-driven app tested in early psychosis has not been reported in the literature. As such, the lack of controlled trials in the field precludes firm conclusions regards feasibility, acceptability, and intervention effects in this group.11

© The Author(s) 2018. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Page 1 of 11

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby032/4939353

by guest

on 23 March 2018


We report here on Actissist, a DHI that is broadly designed to speed up recovery and improve quality of care. The Actissist system is unconstrained by traditional service settings and can be used for active self-management and facilitate shared-decision making about treatment. Grounded in the cognitive model of psychosis, and following an extensive period of co-design with patients and stakeholders, Actissist is more specifically a theory-informed smartphone app targeting key early psychosis domains. The system swiftly identifies and challenges unhelpful appraisals of psychosis-related experiences and provides alternative, more helpful coping strategies in the real-time context of one’s daily life. Actissist was informed by content described in various published academic works.12-21 We draw on experience sampling methodology to prompt participants to engage with the app and build on the clinical protocols described by Granholm et al2 and Ben-Zeev et al.22 Using an agile, iterative process of development, beta-testing and with user experience design (UX) in mind, Actissist targets 5 domains associated with early psychosis relapse: auditory verbal hallucinations; paranoia; perceived criticism; socialization; and cannabis use.

The overarching aim of this Medical Research Council (MRC)-funded trial was to establish proof-of-concept evidence that the Actissist intervention is feasible and acceptable in early psychosis compared with a symptommonitoring app as an active control condition.23 In line with MRC guidelines for developing complex interven-tions,24 the a priori focus of the trial was estimation of treatment effects. The study had 2 aims: (1) test the safety, feasibility, and acceptability of the Actissist intervention; (2) provide preliminary evidence of intervention effects on clinical and functional outcomes. This is the first RCT of a DHI targeting putative mechanisms for early psychosis against an active control condition.

Methods

Study Design

A single blind, proof-of-concept, pilot RCT of 36 early psychosis patients with random allocation using a 2:1 ratio

Page 2 of 11

to receive either Actissist plus treatment as usual (TAU; n = 24) or ClinTouch (a symptom monitoring app) plus TAU (n = 12) over 12 weeks. Trained researchers blind to treatment allocation completed study assessments at time 1 (baseline), time 2 (12-weeks, post-treatment), and time 3 (22 weeks). Eligibility criteria were: (1) in current contact with an EIS in the North West of England; (2) capacity to provide informed consent; and (3) English language proficient. EIS are multidisciplinary community mental health services that provide psychosocial and pharmacological treatment and support to people in their first 3 years of their initial episode of psychosis. Exclusion criteria were: (1) aged less than 16 years at point of recruitment; (2) not capable of giving informed consent; (3) non-English proficient; and (4) inpatient at point of recruitment. Inclusion criteria were as broad as possible to improve the external validity of the trial. The trial was prospectively registered (ISRCTN34966555) and received ethical approval from the National Research Ethics Committee West Midlands—South Birmingham (14/WM/0118).

Recruitment and Randomization

Participants were recruited over 7 months from several NHS Trusts in the North West of England. Health professionals working within EIS identified eligible participants and passed on the contact details of those who consented to be contacted to either a trained researcher or a clinical studies officer (CSO) from the UK Clinical Research Network who supported recruitment to the trial. Following consent to contact, the researcher or CSO invited and consented participants into the trial. Following baseline assessment, participants were randomized in a 2:1 random allocation designed to maximize information about the Actissist intervention. Since hypothesis testing was not the objective of this study, a sample size of 36 was chosen a priori to assess feasibility, conduct preliminary statistical analyses, and obtain parameters to inform a robust power calculation for a fully powered efficacy trial.

Where possible, randomization occurred within 2 working days of baseline assessment. Receipt of the Actissist or control app typically commenced within 2 working weeks of randomization. The study statistician produced a randomization list using random permuted blocks of size 3 and 6. Notification of group allocation occurred using an independent tool (eLabs25; NWEH26), an online research platform that concealed group allocation. The study coordinator was unblinded to treatment allocation and participants were informed about the outcome of randomization from a researcher after the baseline assessment. Many strategies were used to protect blinding, including researchers working on different days to minimize overlap, researchers not being involved in the randomization process, considering room use and diary arrangements, and reminding participants not to disclose group allocation. Group allocation was revealed only to the participant, responsible clinician, baseline research assessor, and project officer. Overall, there was only one blind break in the treatment group; another rater masked to group allocation completed each respective assessment when unblinding occurred. Accordingly, all ratings used for analysis were masked.

Measures

The primary outcome was feasibility, which was assessed in terms of uptake (the proportion of eligible participants consenting to the study), attrition, the proportion of participants completing user, and alert-initiated data entries across participants (>33% data points), and the proportion continuing for 12 weeks (both arms). Acceptability of the Actissist intervention was assessed via participant feedback. Once written informed consent had been obtained, trained researchers administered a battery of secondary outcome measures (details of measures reported elsewhere27). Demographic information was collected as well as measures of frequency, intensity, and distress of psychotic symptoms (Positive and Negative Syndrome Scale, PANsS28; PSYRATS,29 depression (Calgary Depression Scale for Schizophrenia, CDSS30), functioning (Global Assessment of Functioning scale, GAF, APA, 199431; Personal and Social Performance Scale, PSP32), empowerment (Empowerment Rating Scale, ERS33), health status and health-related quality of life (EQ-5D-5L34). Frequency and quantity of alcohol and cannabis use (Timeline Follow Back, TLFB35), perceived criticism from significant others (perceived criticism scale36), medication adherence and attitudes to medication (Medication Adherence Rating Scale, MARS37), and satisfaction with technology38 was also measured.

All measures were administered at each assessment time-point. Participants were reimbursed £20 for completing assessment time-points. Assessors underwent a rigorous training process and received weekly supervision by the chief investigator. Researchers also attended monthly PANSS supervision groups run by experienced senior clinical academics for the duration of the study period. All assessors met departmental reliability standards after pretrial training (mean ICC = 0.89 PANSS total score across the 4 raters), followed by regular supervision in administration, scoring procedures and interrater reliability checks over the course of the trial period.

Procedure

Full details of the procedure are reported elsewhere.27 Briefly, participants in both conditions received a 45-min phone set-up training session focused on basic use of the smartphone (eg, charging the phone; on/off), demonstration of the app (both conditions), setting a passcode, and navigating participants through the app domains and settings. For Actissist, participants watched an in-built video explaining the basic principles of cognitive therapy, the theoretical orientation upon which the Actissist app is based. For ClinTouch, participants were provided a rationale for symptom monitoring. In both apps, participants could also view written and visual “in-app” instructions. Participants had the opportunity to use the app and ask questions. They were instructed to charge the phone regularly, to carry the phone at all times, and to go about their daily life as usual. No restrictions were placed on smartphone use.

Following smartphone demonstration, participants were instructed to use the app for 12 weeks. Participants were instructed to respond to alerts wherever possible and were encouraged to use the on-demand features as and when needed. All participants received a weekly phone call from the project manager to troubleshoot equipment functions. Software was either preloaded on a loaned smartphone with £10 and topped up remotely each month to support data connectivity over the trial period or at their request downloaded on to the participant’s own smartphone. Participants using their own handset were given £10/month to cover data usage costs. Engagement with the apps was incentivized; a £10 shopping voucher was given to participants on a fortnightly basis over the intervention period who completed at least one-third of data entry points. A criterion was applied to determine whether an entry contributed to the overall app engagement algorithm. Specifically, there was a maximum of 3 valid entries per day. To prevent participants from artificially inflating their app usage to achieve the app usage incentives, where a participant self-initiated use (or where there was a combination of self- and prompted-initiation) which exceeded 3 entries per day, only the first 3 entries contributed to the engagement figure. Serious adverse events (SAEs) were strictly monitored, reviewed and documented by the team and discussed with a nominated senior clinical academic and trialist independent to the team.

Interventions

Actissist Actissist is a DHI that the user can engage with spontaneously or in response to being prompted. It then collects responses from the user and wirelessly uploads user responses to a server. Actissist is divided in 2 parts, although presented as a single app. Firstly, at 3 pseudo-randomized time points per day, 6 days a week between 10.00 and 22.00, an auditory alert followed by a visual prompt is emitted from the app inviting participants to access the app. The notifications persist on the handset (ie, no time out) until such point as they are accepted, dismissed, “snoozed” (up to 15 min), or another notification is received. The notifications serve merely as a reminder; the app also allows self-initiated use at any point. If a user accepts a notification or initiates use they are invited to select an intervention domain(s) and then complete a series of self-assessment questions structured as question-answer exchanges that focus on cognitive appraisals, belief conviction, emotions and associated behaviours. Depending on the appraisal selected, the exchange is followed by normalizing messages and cognitive or behavioral strategies aimed at suggesting ways of coping with distressing experiences. Multiple messages and images associated with each exchange minimize boredom and repetition within the app. Alternatively, participants can report that they have had “no problems like this” since their last notification or (self-initiated) interaction. Part 2 includes a menu of multi-media options that act in a stand-alone fashion designed to complement and support the feedback from the intervention domains. This supplementary content contains information and activities including relaxation and mindfulness exercises, recovery stories (videos), a range of fact sheets (eg, low mood, anxiety, self-esteem), external links to web-related content (eg, TED talks), daily diary, and emergency contacts resources. Furthermore, a graphical summary of data points entered over the previous 7 days allows users to track distressing experiences to support active self-management of symptoms and shared decision making about treatment with clinicians. Users can customize the aesthetics of the Actissist interface; for example, personally meaningful images from the smartphone’ s local storage can be set as wallpaper to facilitate positive mood induction. Figure 1 displays a visual schematic of the system, including screenshots of the Actissist app.

ClinTouch (Control Condition) The ClinTouch app is a symptom-monitoring app that triggers, collects, and wirelessly uploads symptom data to a server. As in the treatment condition, the app emits an alarm prompting participants to access the app at 3 pseudo-randomized time points per day, 6 days a week between 10.00 and 22.00 for 12 weeks alongside usual treatment. The ClinTouch protocol is outlined in detail in Palmier-Claus et al23; although, the number of prompts was altered for parity with Actissist alerts, such that participants submit one-and-a-half data points daily with 10 branching items covering positive psychotic symptoms, anxiety, and mood. As each full data point was collected over 2 separate alerts, this equates to having received 3 alerts every day. The alert invites participants to use a touchscreen slider to rate the severity of 12 individual symptoms on a

To minimize risk, we did not store identifying data on either the app or the server. Participants set a passcode



Page 4 of 11

to access the smartphone. Three general principles of information security (confidentiality, integrity, and availability) were followed in the design and implementation of the trial. All data transmitted to and from the servers was encrypted over https with strong ciphers as detailed in the Approved Cryptographic Algorithms Good Practice Guidelines.39

TAU involved regular clinician meetings, medication, risk monitoring, and psychosocial interventions. Actissist and ClinTouch are standalone apps that do not link with services.

Statistical Analysis

Analyses follow the CONSORT 2010 Statement,40 showing referral and attrition (ie, participant flow) and an a priori analysis plan was published.27 Analyses were undertaken in Stata (version 14.1) after completion of the endpoint assessment. The primary outcome (feasibility) was assessed in terms of uptake (the proportion of eligible participants consenting to join the study), attrition, proportion of participants completing user and alert-initiated data entries across participants, and the proportion continuing for 12 weeks (both arms). Demographic data were presented using descriptive statistics. Ecological momentary interventions typically operationalize >33% data points completed as evidence of compliance,41,42 which was the “accept” criterion for compliance that we adopted. The “target” criterion was 50% of participants submitting 50% of data entries. Linear regression was used to examine the effect of random allocation on the secondary outcomes at 12 and 22 weeks separately, adjusting for outcome measures at baseline. We report adjusted mean differences and their standard errors, Cohen’s D standardized effect sizes and their corresponding bootstrapped 95% confidence intervals (CIs), based on adjusted mean differences and the pooled standard deviation at baseline. Acceptability was assessed by participant feedback using a semi-structured interview. Feedback was thematically organized.

Results

Sample Characteristics and Feasibility Outcomes

A summary of demographic and clinical information for all participants is displayed in tables 1 and 2. There were very few drug users in the sample (2 cannabis, 2 cocaine, 1 mephodrone, 1 gogaine); 14/36 participants smoked cigarettes regularly. As can be seen in figure 2, uptake into the trial was high: 38/59 people (64.4%) referred participated to the full trial. Reasons for declining participation included medication side effects, concentrating on studies/ employment, and not wishing to focus on mental health at the current time. Retention in the Actissist arm and tolerance for the Actissist app was excellent, as evidenced by no participant withdrawals. The “accept” criterion for

Table 1. Demographic Characteristics: Means (SD) or Numbers (%) of Participants

Actissist (n = 24)

ClinTouch (n = 12)

Age at first symptoms

20.21 (7.37)

18.33 (7.00)

Sex

Male

15 (62.5)

3 (25.0)

Female

9 (37.5)

9 (75.0)

Ethnicity

White British/Irish

21 (87.5)

10 (83.3)

Black Caribbean/African

2 (8.3)

2 (16.7)

Asian

1 (4.2)

0

Medication

Yes

17 (70.8)

11 (91.7)

Not known

7 (29.2)

1 (8.3)

Psychotherapy

Yes

5 (20.8)

0 (0.0)

No

8 (33.3)

5 (41.7)

Not known

11 (45.9)

7 (58.3)

Years of education

13.69 (2.72)

13.42 (2.62)

Marital status

Single

18 (75.0)

10 (83.3)

Married or partnership

0 (0.0)

1 (8.3)

Co-habiting

6 (25.0)

1 (8.3)

Employment status

Employed

6 (25.0)

3 (25.0)

Education/training

8 (33.3)

2 (16.7)

NEET

10 (41.7)

7 (58.3)

Previous admissions

Yes

9 (37.5)

3 (25.0)

No

15 (62.5)

9 (75.0)

Note: NEET, not in education, employment, or training; PANSS, Positive and Negative Syndrome Scale.

data points completed was met in both trial arms (75% and 50% participants, respectively, submitting >33% data entries as per our pre-specified criteria). In other words, 75% of Actissist participants used the app on average at least once a day over the 12-week intervention period, suggesting excellent engagement and acceptability of the Actissist app. The “target” criterion was achieved in the Actissist arm only (63% vs 42% in ClinTouch). All participants except one (97%) remained in the trial (both arms) until the end. The participant who withdrew from the study returned the phone and withdrew from EIS all together (nonresearch related incident). No research-related SAEs were recorded for any participants during the study period, suggesting that both apps are safe. Completion of assessments was also high: 72% (26/36) and 83% (30/36) participants successfully followed up at post-treatment and 22 weeks, respectively.

General Pattern of App Usage

Participants mostly interacted with the app in the latter part of the day and earlier in course of the 12-week intervention (although the target engagement rates were still achieved for the intervention period). The most popular prompted domain was voices (481 entries), followed

Table 2. Clinical Measures at Baseline, by Randomized Group

Measure

Baseline

ClinTouch (N = 12)

Actissist (N = 24)

Mean

SD

Mean

SD

PANSS positive

17.8

5.9

16.0

3.9

PANSS negative

12.8

2.5

15.2

4.0

PANSS general

34.0

6.4

34.9

7.6

PANSS total

64.6

11.1

65.9

12.9

Calgary—mild

1.8

1.4

2.4

1.3

Calgary—moderate

1.7

1.4

1.3

1.2

Calgary—severe

2.2

1.7

1.5

1.8

Calgary—total

11.7

5.1

9.1

5.4

PSYRATS—delusions

12.5

8.8

11.9

7.3

PSYRATS—AH

13.3

15.2

16.6

14.3

PSP

51.3

13.7

48.9

11.5

GAF functioning

53.2

13.9

50.5

11.5

GAF symptoms

48.2

14.0

48.5

13.0

GAF total

46.4

14.3

44.4

10.4

PCS

19.1

8.4

20.9

8.1

ERS

77.8

8.7

82.3

7.7

EQ5D likert 0-100

58.8

21.1

64.0

16.9

Average alcohol consumption (past 30 days)

33.36 (n = 9 alcohol users)

45.21

23.45 (n = 18 alcohol users)

26.38

Number of nonsober days (past 30 days)

5.50

6.67

3.29

4.75

MARS

14.80

2.10

15.67

1.98

Note: PANSS, Positive and Negative Syndrome Scale; Calgary, Calgary Depression Scale for Schizophrenia; PSYRATS, Psychotic Symptoms Rating Scale; PSP, Personal and Social Performance Scale; GAF, Global Assessment of Functioning Scale; PCS, Perceived Criticism Scale; ERS, Empowerment Rating Scale; EQ5D, EuroQol-5D-5L; MARS, Medication Adherence Rating Scale.


by suspicious thoughts (404 entries), socializing (220 entries), criticism (157 entries), and cannabis (31 entries). The most popular unprompted domain was suspicious thoughts (97 entries), followed by socializing (65 entries), voices (64 entries), criticism (39 entries), and cannabis (14 entries). Considering both prompted and unprompted entries, voices overall was the most frequently accessed domain overall (545 entries), followed by suspicious thoughts (501 entries), socializing (285 entries), criticism (196 entries), and cannabis (45 entries). Expressed as an average per participants across each domain over the intervention period, participants clicked on average 1.85 times (range: 0-11) on the cannabis domain (2 outliers noted, both clicking 11 times), 1.88 times (range: 1-50) on the socializing domain (2 outliers noted, clicking 40 and 50 times), 20.88 times (range: 0-94) times on the suspicious thoughts domain (1 outlier noted, clicking 94 times), 8.17 (range: 1-25) times on the criticism domain (1 outlier noted, clicking 31 times), and 22.71 times (range: 0-174) times on the voices domain (5 outliers noted, clicking 51, 54,, 58, 74, and 174 times). These findings show that while voices was the most frequently accessed domain overall, this seemed to be influenced by a few participants who interacted with this domain quite intensively. The remaining domains appeared less influenced by outlier responses. Participants lost 2 phones during the trial. The majority of participants used a study phone

Page 6 of 11

(n = 31). There was no difference in those who used their own phone (12.5% Actissist; 8.3% ClinTouch) versus those who were loaned a study phone (83.3% Actissist; 91.7% ClinTouch).

Secondary Outcomes

Summary statistics for all secondary outcomes across time points and conditions are shown in table 3. Inspection of the effect sizes and confidence intervals suggest that there were improvements in key outcome measures, including PANSS negative score, general and total scores, and Calgary (mild, moderate and total) scores in the Actissist group relative to the control group post-treatment. The regression coefficients (ie, adjusted mean differences) and standardized effect sizes (Cohen’s D) are numerically higher on these variables at the post-treatment Actissist assessment. This suggests that participants understood the content of the app and learnt new skills, improving general psychotic symptoms and mood in the short-term. Effects were not fully sustained at 22-week follow-up; although, there was no decline on any of the clinical outcomes measured.

Acceptability

The Actissist system was acceptable, enjoyable, beneficial, and easy to use (see supplementary table 1); 90%


Fig. 2. CONSORT diagram.

participants said they would recommend Actissist to others in a similar position. Illustrative feedback quotations are provided in table 4.

Discussion

This is the first study to evaluate the feasibility, acceptability, and safety of a theory-driven smartphone app in a randomized controlled design compared against an active control condition in early psychosis. Benefits reported for psychosis outcomes were observed in comparison to a similar smartphone app platform for tracking mental health in psychosis. Specifically, we showed that Actissist was feasible, acceptable, and safe, with high levels of satisfaction and indications of a beneficial effect, even over an active control group (symptom monitoring). Retention in the study was excellent (35/36 participants; 97%), which reflects other DHI retention rates of around 92% psychosis patients remaining in the trial until the end.11 Assessment completion was high and reflects similar DHI trials in psychosis. Engagement with the system was high (75% participants used the app at least once/ day), which reflects engagement with other alert-based DHIs in psychosis (average response rate to alerts across trials of 71.9%, with 86.5%-94% participants interacting with the app on the predefined study days).11 Feedback from participants was overwhelmingly positive, suggesting that participants enjoyed using the app, understood the content of the app, with some suggestion that participants implemented new skills in the course of their day-to-day lives, evidenced by the promising treatment effect estimates. Findings sit alongside those emerging in the psychosis literature, which shows that digital symptom monitoring systems are feasible and acceptable to patients.2,22,23,43

The treatment effects immediately post-treatment favoured the Actissist group, and were large. This suggests that the Actissist app conferred added benefit over and above routine symptom monitoring in terms of negative, general and total psychotic symptom scores and mood in the short term. Although there was no decline on any of the clinical outcomes measured at 22-week follow-up, treatment estimates were not maintained at this timepoint, suggesting that further testing of sustained effects over time is needed. Furthermore, participants tended to use the app later in the day, reflecting the importance of using personalized alerts rather than pseudo-random alerts over a prespecified time period within days.

Further research should be hastened in light of digital health care initiatives that lack an evidence base,

Table 3. Summary Statistics and Treatment Effects at Post-Treatment

Measure

Post-Treatment

Effect (SE)

95% CI

Cohen’s D; 95% CI

Post-Treatment Scores

ClinTouch (N = 8)

Actissist (N = 18)

Mean

SD

Mean

SD

PANSS positive

14.5

5.1

13.0

3.8

-1.30 (1.29)

-3.97, 1.37

-0.28 (-0.85, 0.29)

PANSS negative

14.0

3.9

13.3

4.5

-3.04 (1.26)

-5.64, -0.44

-0.85 (-1.58, -0.12)

PANSS general

34.5

8.7

28.4

8.8

-6.23 (2.04)

-10.45, -2.00

-0.86 (-1.44, -0.28)

PANSS total

63.0

15.6

54.7

14.6

-10.47 (3.54)

-17.80, -3.14

-0.85 (-1.44, -0.25)

Calgary—mild

2.9

1.1

1.9

1.5

-1.22 (0.58)

-2.42, -0.01

-0.92 (-1.83, -0.01)

Calgary—moderate

4.0

2.8

1.4

1.8

-2.42 (0.91)

-4.31, -0.54

-1.92 (-3.42, -0.43)

Calgary—severe

3.8

3.8

1.3

2.6

-1.92 (1.26)

-4.52, 0.68

-1.09 (-2.56, 0.39)

Calgary—total

10.8

5.1

5.1

5.1

-3.43 (1.61)

-6.76, -0.11

-0.65 (-1.28, -0.02)

PSYRATS—delusions

10.9

9.9

7.8

7.2

2.15 (3.11)

-4.30, 8.60

0.28 (-0.55, 1.1)

PSYRATS—AH

5.3

10.4

16.5

14.7

-3.07 (2.71)

-8.66, 2.54

-0.21 (-0.59, 0.17)

PSP

48.0

12.0

53.5

15.1

5.77 (4.07)

-2.64, 14.18

0.47 (-0.22, 1.16)

GAF functioning

52.8

17.7

53.8

16.3

1.02 (5.43)

-10.22, 12.26

0.08 (-0.83, 1)

GAF symptoms

54.3

16.0

57.8

15.1

3.74 (5.05)

-6.72, 14.19

0.28 (-0.5, 1.07)

GAF total

49.9

15.5

49.3

13.6

0.85 (4.87)

-9.22, 10.91

0.07 (-0.78, 0.92)

PCS

22.3

8.8

20.2

5.9

-2.13 (2.94)

-8.21, 3.96

-0.26 (-1, 0.48)

ERS

81.2

2.1

86.2

5.8

3.47 (1.95)

-0.60, 7.54

0.43 (-0.07, 0.94)

EQ5D5L likert 0-100

40.0

26.0

71.1

21.3

-117.17

-283.44, 49.10

-6.38 (-15.43, 2.67)

MARS

14.33

2.66

15.12

1.93

0.37 (0.98)

-1.67, 2.41

0.18; -0.82, 1.19

Average alcohol unit

4.29 (n = 8)

3.45

8.64 (n = 18) 13.19

1.45 (4.10)

-7.04, 9.94

0.30; -1.45, 2.04

consumption over the nonsober days (last 30 days)

22 Week Scores

ClinTouch (N = 9)

Actissist (N =

21)

Effect (SE)

95% CI

Cohen’s D; 95% CI

Mean

SD

Mean

SD

PANSS positive

16.2

5.40

13.2

4.6

-1.90 (1.48)

-4.93, 1.14

-0.41 (-1.06, 0.24)

PANSS negative

13.8

4.21

13.8

4.9

-2.73 (1.33)

-5.46, 0.003

-0.76 (-1.53, 0)

PANSS general

33.6

10.14

28.9

7.5

-4.84 (2.69)

-10.37, 0.68

-0.67 (-1.43, 0.09)

PANSS total

63.6

17.54

52.9

17.4

-11.27 (6.78)

-25.19, 2.65

-0.91 (-2.04, 0.21)

Calgary—mild

2.7

1.58

2.4

1.2

-0.36 (0.56)

-1.50, 0.78

-0.27 (-1.13, 0.59)

Calgary—moderate

2.0

2.45

1.8

2.1

-0.21 (0.91)

-2.08, 1.66

-0.17 (-1.65, 1.32)

Calgary—severe

2.7

3.16

1.6

3.1

-0.44 (1.12)

-2.74, 1.86

-0.25 (-1.55, 1.06)

Calgary—total

7.3

4.82

6.0

4.9

0.57 (1.60)

-2.71, 3.85

0.11 (-0.51, 0.73)

PSYRATS—delusions

11.3

7.5

8.0

7.8

-2.11 (2.50)

-7.24, 3.02

-0.27 (-0.93, 0.39)

PSYRATS—AH

11.8

13.4

16.8

14.3

3.30 (3.95)

-4.81, 11.40

0.23 (-0.33, 0.78)

PSP

54.3

15.5

56.9

14.2

3.24 (5.49)

-8.01, 14.50

0.26 (-0.65, 1.18)

GAF functioning

57.3

10.3

59.4

14.2

3.23 (4.69)

-6.39, 12.84

0.26 (-0.52, 1.04)

GAF symptoms

49.8

17.4

55.5

16.6

5.40 (6.26)

-7.45, 18.25

0.41 (-0.56, 1.37)

GAF total

48.2

14.7

52.0

16.2

4.27 (6.27)

-8.60, 17.14

0.36 (-0.73, 1.45)

PCS

22.6

7.4

21.2

7.2

-1.54 (2.98)

-7.66, 4.58

-0.19 (-0.94, 0.56)

ERS

82.4

6.6

85.0

7.5

-0.92 (1.98)

-5.00, 3.16

-0.11 (-0.62, 0.39)

EQ5D5L likert 0-100

56.9

18.0

63.1

21.2

4.34 (6.93)

-9.89, 18.59

0.24 (-0.54, 1.01)

MARS

13.29

1.80

15.59

2.09

0.63 (0.84)

-1.12, 2.38

0.31; -0.55, 1.18

Average alcohol unit

4.15 (n = 9)

3.91

4.52 (n = 21)

4.98

-0.14 (1.84)

-3.92, 3.64

-0.03; -0.80, 0.75

consumption over the nonsober days (last 30 days)

Note: PANSS, Positive and Negative Syndrome Scale; Calgary, Calgary Depression Scale for schizophrenia; PSYRATS, Psychotic Symptoms Rating Scale; PSP, Personal and Social Performance Scale; GAF, Global Assessment of Functioning Scale; PCS, Perceived Criticism Scale; ERS, Empowerment Rating Scale; EQ5D, EuroQol-5D-5L; MARS, Medication Adherence Rating Scale.

and future consideration evaluating fast-paced technological innovations outside an RCT context is needed. There is a need to ensure parity and to limit the exclusion of low-income individuals who cannot afford smartphones and their associated cost. This study has notable strengths. We used an active control symptom-monitoring

Page 8 of 11

Table 4. Participant Feedback (n = 15)

Nature of Feedback

Illustrative Quotation

Positive views about the Actissist app

Ease of access

“. that app what it does, it says ‘I’ve got a CPN in my pocket, I’ve got a care provider in my pocket that I can, I can go out quite freely now without my CPN I don’t have to arrange something with my CPN . It’s kind of, it gives you a bit of freedom to say ‘hold on a second, I don’t have to wait for my CPN.” (Participant 9)

“I read the app before I went to the party, then, when I got to the party, I was in there about half an hour, twenty minutes, in, the voices I started leaving to go to the door, I wanted to get out, again, the app came into its own, I said, ‘can I just nip to the toilet quickly?’ just went to the toilet, just took out the app, just had a quick read, quick reassurance, back into the party.” (Participant 9) “It’s accessible, you can use it anywhere, erm in any situation, it wouldn’t be like oh you’ve got to go to the doctors or anything like that ... you can deal with it straight away.” (Participant 10)

“If you feel, if you, if someone feeling so low and so depressed like I was .you wouldn’t want to talk to someone about those thoughts, cos they were disturbing and having that app there, just ready, like, cos it beeps, cos it beeps, whenever, every couple of hours. It was just perfect, it’s like an immediate help.” (Participant 106)

Inspires confidence and empowerment

“It’s like having somebody in the room who know’s what they’re talking about . putting confidence into you.” (Participant 132)

“In mental health you feel a little bit like a criminal, criminalized sometimes and I think with it being on the phone it’s in your hands a little, it’s under your control a bit more, as opposed to feeling a bit like you’re under house arrest.” (Participant 11)

Facilitates self-management

“. you become your own therapist and that’s what CBT is about, being able to change your behavior . reassess a situation, about going forward on your own, uhm solution.” (Participant 5)

Becomes part of your routine

“Noticing it in an app like that, and in that order, yeah it, it’s encouraging. So you tend to get in a routine with it, which is good, or I did . and as part of your daily routine it’s like as if sommat’s looking after you, in a way, which is good.” (Participant 109)

“. it did start to feel part of my normal routine . it was good, it was sort of like having a buddy [laughs] um so yeah every time it sort of asked you to check in it was quite a good feeling.” (Participant 7)

“It was different, it wasn’t something I was used to, erm, and for me, it was quite good ‘cause, I kind of, I only see my care coordinator once a week, sometimes I just, I don’t like, do what she tells me but it’s like a reminder. So it kinda fits in with that for me a bit, fitted with that for me as well.” (Participant 107)

Ideas for improvement Minimize repetition and personalize content Personalizing alerts to fit with lifestyle

“Sometimes you can get annoyed with, a bit sick of these questions, that’s all, but it’s just sometimes ’cos, ’cos you’ve heard it before, that’s all, that’s all.” (Participant 109)

“. it seemed like it’s prompted me too often.” (Participant 128)

I didn’t like it when it reminded me to do it, I could do it off my own accord, when I knew I needed to do something to kill time or just to get like information out of it. Erm but, the constant reminder of it, it was just like nooo....” (Participant 111)

Depth and variety of content

“I found it was helpful at first but then I found content on the actual app was too limited. I think, there’s only so many answers, so when you answer like a question, there’s only so many like responses it can give you.” (Participant 128)

app condition, which matched the overall look-and-feel and functionality of the Actissist app, thus accounting for the nonspecifics of smartphone use. The apps were different, however, in that ClinTouch is designed to be completed in-the-moment and cannot be left to complete at a chosen time, whereas Actissist acts as more of an active and responsive self-management tool and is therefore available whenever the user requires (either prompted or self-initiated). We implemented a rigorous reporting SAE procedure for both groups. Trials without an active control condition have problems gathering a true measure of SAEs as researchers typically have less contact with the TAU group than the treatment arm. Finally, raters were blind to group allocation and stakeholders were co-designers of the system. There are some limitations. Participants were incentivized to use both apps, which may have increased usage beyond what would be observed in a real-world setting. Second, we operationalized engagement according to experience sampling methodology criteria (completion of 33% data entries over the intervention period). As this study was incentivized, we needed to apply a criterion to prevent participants from artificially inflating app usage to obtain incentives. In future DHIs, we suggest researchers report app usage more descriptively rather than prespecify completion rates.44,45 Finally, whilst the small sample could impact generalizability of findings, participants were representative of a help-seeking early psychosis group.

Our findings suggest that the Actissist system may confer additional benefits over routine mobile symptom monitoring in the short term. Participants were engaged, active, and adherent with the system; therefore, findings justify proceeding to a fully powered trial. This study represents an important and significant step toward developing a technology platform for delivering a range of psychosocial interventions for psychosis. Indeed, this study is the first to show that an active self-management app can potentially improve outcomes in psychosis, even beyond a passive symptom-monitoring app. It shows how proof-of-concept trials can underpin digital experimental health care with empirically derived theoretical frameworks. If trials such as Actissist are effective, a major challenge is for mental health services to recognize and incorporate DHIs into the health care setting.

Supplementary Material

Supplementary data are available at Schizophrenia Bulletin online.

Acknowledgments

Thanks to the service users, clinical teams, and our Expert Reference Group for their support, advice, and participation in the study. Thanks to Susannah James for supporting the day-to-day running of the trial.

References

Page 10 of 11

documents/pdf/rcts-for-complex-interventions-to-improve-health/. Accessed December 21, 2017.

Practitioner's Corner

Efficacy of Baduanjin Exercise and Feasibility of Mobile Text Reminders on Follow-up Participation in People With Severe Mental Illness: An Exploratory Study

MING-DE CHEN, PhD YA-CHIN YEH, MS YI-JUNG TSAI, PhD YEN-CHING CHANG, PhD JUNE-WEI YU CHING-HUI HSU, MS


Background: People with severe mental illness (SMI) frequently have poor physical health, which can in part be related to a low level of physical activity. The goal of this exploratory study was to examine the efficacy of a group participating in Baduanjin (a type of traditional Chinese exercise) on the health of individuals with SMI and the feasibility/ acceptability of using short message service (SMS) reminder strategies to prompt continued exercise during a follow-up period. Method: Participants (N=11) participated in a Baduanjin group session for 8 weeks and then maintained home-based Baduanjin with SMS reminders for another 8 weeks. Physical and psychological tests were administered to participants.

Results: Significant improvements were found in balance and processing speed and in some domains of the 36-Item Short Form Health Survey (SF-36) after the Baduanjin program. Participants were able to maintain engagement in 80% of the expected practice time during the follow-up period. The acceptability of strategies to support home-based exercise, including SMS reminders, was high.

Conclusions: This preliminary study suggested the efficacy of Baduanjin and the feasibility of SMS reminders in maintaining follow-up participation in people with SMI. Future studies using a larger sample size and a control group are needed to confirm the findings.

(Journal of Psychiatric Practice 2016;22;241-249)

KEY WORDS: mental illness, Qigong, Baduanjin, exercise, short message service, text message

People with severe mental illness (SMI) have been found to have poorer physical health than the general population.1 Research has found that physical activity has the potential to improve the physical and mental health of people with SMI.2,3 However, compared with the general population, people with SMI are less physically active.4-6 Many factors, such as low self-efficacy, fatigue, a low level of social support, and poor motivation, influence this group's regular participation in physical activity.4,7,8 To improve this situation, Wolff et al9 provided several recommendations, including choosing a less cognitively and physically demanding activity as well as providing professional supervision and training management.

Qigong, a type of physical activity that originated in Chinese culture, involves low to moderate intensity aerobic exercise and has been found to have physiological, cognitive, and psychological benefits.10,11 Tai-chi, a well-known type of Qigong, has been criticized as being too complicated for beginners to learn and practice independently.12 Baduanjin is another type of traditional Chinese Qigong exercise that is less physically and

CHEN, TSAI, and YU: Department of Occupational Therapy, Kaohsiung Medical University, Kaohsiung City, Taiwan; YEH: Department of Occupational Therapy, Shu-Zen Junior College of Medicine and Management, Kaohsiung City, Taiwan; CHANG: Department of Occupational Therapy, National Cheng Kung University, Tainan City, Taiwan; YU: Department of Psychiatry, Chung-Ho Memorial Hospital, Kaohsiung Medical University, Kaohsiung City, Taiwan; HSU: Division of Physical Education, Kaohsiung Medical University, Kaohsiung City, Taiwan

Ya-Chin Yeh, MS, and Yi-Jung Tsai, PhD, were equal contributors to this manuscript.

Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.

Please send correspondence to: Ching-Hui Hsu, MS, Division of Physical Education, Kaohsiung Medical University, 100, Shih-Chuan 1st Road, Kaohsiung City 80708, Taiwan (e-mail: hsu88@kmu.edu.tw).

Supported in part by the Department of Sports, Kaohsiung City, Taiwan, and by the Ministry of Science and Technology, Taiwan (MOST- 102-2410-H-037-014).

The authors declare no conflicts of interest.

DOI: 10.1097/PRA.0000000000000158 cognitively demanding and can be learned easily and practiced without restrictions on time and space.13,14 Previous research has demonstrated that it is feasible to offer Baduanjin as a communitybased physical exercise program for the elderly.15 Baduanjin has also shown health benefits, such as improvements in physical fitness and quality of life, in the general population.16,17 However, information about the impact of Baduanjin on people with SMI is limited. Deficits in processing speed have been shown to be one of the prominent cognitive impairments in people with psychosis.18 Previous research found that Qigong exercise could improve cognitive function in patients with cancer,10 suggesting that it would be worthwhile to investigate the effects of Baduanjin on processing speed in individuals with SMI. Baduanjin could potentially be a valuable exercise for improving physical and psychosocial functioning in this population,

It is challenging to improve adherence to physical activity in individuals with SMI,19,20 and choosing the appropriate type of exercise can play a crucial role in enhancing motivation. Studies have suggested that physical activity should be continued on a regular basis to maintain its positive effects on health; otherwise, a significant reduction in health-related outcomes would be found 2 weeks to 2 months after discontinuation of physical exercise.21,22 Although structured exercise programs with instructors are an easier method for therapists to ensure participantssafety and consistent levels of physical activity, they involve some disadvantages, including costly staffing, less flexibility in practice time, and restricted access to the exercise facility.2 Therefore, it is more practical to find motivational strategies that can maintain participation in the physical activity after the individual has completed the structured exercise program.

Emerging evidence supports the use of mobile phone and communication technology to improve the quality of interventions for psychiatric disorders.23-25 In particular, strategies based on using a short message service (SMS) have been found to provide a significant advantage in enhancing adherence to antipsychotic medications in patients with schizophrenia26 and in increasing levels of physical activity in older adults27 and physically inactive veterans.28 An SMS-based strategy to promote home-based exercise has the merits of low cost, minimal restrictions on time, and high accessibility to opportunities to practice physical activity. It also offers social support to strengthen motivation to participate in physical activities.

The objectives of this study were (1) to examine the efficacy of an 8-week structured Baduanjin exercise group on physical and mental health in people with SMI; and (2) to examine the feasibility and acceptability of using SMS reminder strategies to prompt continued exercise during an 8-week follow-up period.

METHODS

Design and Participants

This study included 2 phases. During phase 1, participants completed a structured 90-minute Baduanjin exercise group session twice a week for 8 weeks. During phase 2, participants completed a home-based Baduanjin program with SMS reminders for another 8 weeks. The study protocol was approved by the Institutional Review Board of the research institute.

Participants were recruited from 3 communitybased psychiatric rehabilitation centers. Inclusion criteria included a diagnosis of schizophrenia, major depressive disorder, or bipolar disorder by psychiatrists on the basis of criteria in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision; being a user of a communitybased psychiatric rehabilitation program; being between 20 and 64 year of age; owning a personal mobile phone; and having a stable psychiatric condition (defined as no changes in antipsychotic medications or mood stabilizers during the previous 3 mo).29 Individuals with physical and cognitive conditions that prevented participation in Baduan-jin exercise or who were currently enrolled in studies evaluating other interventions were excluded. The community-based psychiatric rehabilitation center is an institute that provides mental health rehabilitation programs for people with mental illness living in the community. Services may include training in social skills and/or activities of daily living, vocational rehabilitation programs, and health promotion programs. Flyers were distributed at research sites. Service users who were interested in this study were instructed to contact staff or the researchers. People who met the inclusion criteria were recruited. Their


participation was voluntary and informed consent forms were obtained from all participants.

Intervention

The Baduanjin exercise group was led by one of the researchers (C.-H.H.) who has years of experience teaching Baduanjin exercise. Study participants practiced Baduanjin exercise in a group with the researcher for 90 minutes per session, twice a week, for 8 weeks. The entire set of Baduanjin includes 8 sections (Fig. 1). During the first 4 weeks, participants learned and practiced each section of Baduanjin exercise under the researcher's guidance. During the following 4 weeks, participants practiced each section as well as the whole set of Baduanjin exercise.

After completing the Baduanjin exercise group, each participant was encouraged to perform Baduanjin exercise at home for at least 30 minutes per day, 3 times a week, for another 8 weeks. To reinforce their adherence, each participant was given a home-based exercise package and daily record sheet and received SMS reminders. The home-based exercise package included a picturebased educational brochure, video CD/DVD, and soundtrack CD demonstrating the Baduanjin exercise. Two SMS reminders were sent weekly to prompt participants to perform Baduanjin exercise at home. Each participant received the same text messages. Examples of messages included the following: It is getting cool outside. Doing Baduanjin exercise would improve your blood circulation and health condition. Don't forget to mark on the record sheet once done and turn it in to the staff every week.and Hi, you can practice Baduanjin in a little free time. Even if only for 10 minutes, it still counts.The days we sent the text reminders were randomly selected. All of the participants had a personal mobile phone on which they could receive text messages. The researchers also provided a training session on how to check and read messages to ensure all participants were able to receive SMS texts. Finally, participants were asked to mark the daily record sheet after successful completion of each exercise practice.

Measures

Physical and psychosocial functioning were assessed at baseline and after the 8-week Baduan-jin exercise group by independent assessors. Physical tests included anthropometric measurements, blood pressure, aerobic fitness, flexibility, muscular endurance, and balance. Body weight was measured in light clothes and bare feet using a digital scale (Tanita BC-706, Tokyo, Japan) with an accuracy of 0.1 kg. Body mass index was determined by weight (kg)/height (m)2. Waist and hip circumference were taken with a nonstretch tape and calculated as waist-to-hip ratio.30 Aerobic fitness was assessed by a 6-minute walk test.31 Flexibility was measured by a sit-and-reach test, and muscular endurance was measured by the Eurofit 30-second Sit-Up Test.32 Balance function was assessed by a 1-leg balance test with eyes open and eyes closed. Participants were asked to maintain balance as long as possible or until 30 seconds had passed.33 Psychosocial function tests included processing speed and quality of life as determined by the Wechsler Adult Intelligence Scale Digit SymbolCoding test and the 36-Item Short Form Health Survey (SF-36), respectively. After the follow-up phase, a survey developed by the study researchers was conducted to measure the acceptability and frequency of use of the home-based exercise package and the SMS reminders.

Data Analysis

Because of the small sample size, the nonparametric Wilcoxon signed-rank test was used to examine changes in the variables of interest. Change between the assessments at baseline and after participation in the exercise group was considered to represent the efficacy of Baduanjin exercise on health outcomes. To evaluate the feasibility of and adherence to the follow-up homebased Baduanjin exercise phase, the frequency and duration of exercise that each participant performed were retrieved from the daily record sheets. Descriptive statistics were derived from the survey conducted after the follow-up period to characterize acceptability and frequency of usage.

RESULTS

Characteristics of the Participants

Initially 16 participants were enrolled in the baseline assessment, 11 of whom (69%) completed the whole study. Five subjects dropped out, 3 due to a time conflict with work and 2 due to unstable psychiatric symptoms. The 11 participants who completed the study included 8 men and 3 women between 24 and 57 years of age (mean age=38.9y, SD=9.6y). Most of the participants (82%) were diagnosed with schizophrenia. All of the participants had been hospitalized for mental illness, and the majority (54.6%) were overweight or obese. The demographic characteristics of the participants are summarized in Table 1.

Preliminary Efficacy of the Baduanjin Exercise Group

Table 2 shows the changes on physical and psychosocial functions between time 1 (before the Baduanjin exercise group intervention) and time 2 (after the intervention). The results showed no significant change on most physical outcomes, except for performance on 1-leg balance with eyes open (z=2.02, P=0.04). In terms of psychosocial health, significant improvements were found on the Digit Symbol-Coding test (z=2.30, P=0.02) and some domains of the SF-36, including physical role functioning (z=2.54, P=0.01), vitality (z=2.16, P=0.03), social role functioning (z=2.21, P=0.03), and mental health (z=2.10, P=0.04).

Feasibility, Adherence, and Safety

A total of 16 Baduanjin exercise group sessions were conducted, with participants attending 11.5 sessions on average, for an average session attendance rate of 72%. During the 8-week follow-up

TABLE 1. Demographics of Participants (N=11)

Variables

Mean+SDIN (%)

Age (y)

38.9±9.6

Sex

Male

8 (72.7)

Female

3 (27.3)

Education (y)

<9

2 (18.2)

10-12

3 (27.3)

13-16

5 (45.5)

>16

1 (9.1)

Diagnosis

Schizophrenia

9 (81.8)

Affective disorder

2 (18.2)

Body mass index (BMI)*

Normal (18.5<BMI<24)

5 (45.5)

Overweight (24>BMI<27)

4 (36.4)

Obese (BMI>27)

2 (18.2)

Smoking

Yes

2 (18.2)

No

9 (81.8)

Prior hospitalization

Yes

11 (100)

No

0

^Criteria defined by the Ministry

of Health and Welfare in

Taiwan.

practice time on those days was 17 minutes (range, 10 to 31 min). Participants exercised on an average of 24.3 days of the 40 days on which they did not receive text reminders, for a participation frequency of 61% (range, 30% to 100%), and the average daily practice time on those days was also 17 minutes (range, 10 to 31 min). No adverse events were reported throughout the whole 16-week intervention.

Acceptability and Frequency of Use of Home-based Exercise Support Materials

Ten participants completed the survey at the end of the 16-week study (Table 3). The majority of participants (80%) agreed that the support materials as a whole were helpful to them in doing their Baduanjin exercise at home, with participation in the Baduanjin exercise group, the picture-based brochure, and the SMS reminders perceived as the most helpful prompts. Over 70% did not feel bothered by reading the SMS reminders and marking the daily record sheets. Participants reported that the support materials they used most frequently were the SMS reminders and the picture-based educational brochure; the CD and DVD were used less frequently.

phase, participants were asked to perform individual home-based Baduanjin exercise prompted with SMS. In general, the follow-up participation rate was the highest during the first week and declined gradually. During weeks 4 to 8 of the follow-up period, the participation rate fluctuated. On average, participants engaged in Baduanjin exercise 4.6 days per week, with adherence to exercise ranging from 3.9 to 5.2 days per week (Fig. 2A). During the follow-up period, the average weekly participation duration was 73 minutes (range, 59 to 92 min/wk) (Fig. 2B). Thus, with text message reminders, participants were able to maintain partial adherence to Baduanjin exercise to attain 81% of the targeted weekly duration of 90 minutes (range, 66% to 102% of the targeted duration).

Participants exercised on an average of 11.2 days of the 16 days on which they received SMS reminders, for a participation frequency of 70% (range, 44% to 100%), and the average daily

DISCUSSION

This exploratory study suggested that an 8-week Baduanjin exercise group could improve balance function, processing speed, physical role functioning, vitality, social role functioning, and mental health in people with SMI. Participants were also able to continue engaging in over 80% of targeted exercise time with the reinforcement of an SMS strategy for 8 weeks after completing the structured exercise group. The acceptability of home-based exercise support materials was high. A picturebased educational brochure and SMS reminders were the materials that participants used most frequently to continue exercising at home. To the best of our knowledge, this is the first study that has used mobile technology (ie, SMS reminders) to maintain participation in physical activity in people with SMI.

Given the poor health status in this clinical group, the study results add to empirical evidence

TABLE 2. Scores and Change on Physical and Psychosocial Outcome Measures (N=11)

Measure

Time 1 Baseline

Time 2 Postintervention

Statistics

Mean±SD

Mean±SD

Z

P

Physical outcome measures

Body measurements

Weight (kg)

67.5±12.5

67.5±11.8

0.20

0.84

BMI (kg/m2)

24.8±5.5

24.8±5.3

0.15

0.88

WHR

0.9±0.1

0.9±0.1

-1.1

0.29

Blood pressure

Systolic (mm Hg)

114.4±13.8

113±11

-0.41

0.68

Diastolic (mm Hg)

78.8±10.4

75.8±11.7

-1.17

0.24

Physical fitness

6MWT (m)

516.3±80.7

496.9±50

-0.45

0.66

Sit and reach (cm)

12.5±10.8

14.4±8.2

0.14

0.89

Sit-up (N)

8.8±4.1

10.6±3.7

1.73

0.08

OLB-OE (s)

22.8±9.6

28.5±4.3

2.02

0.04*

OLB-CE (s)

12.6±11.2

9.2±7.9

-1.48

0.14

Psychosocial outcome measures

Processing speed

Digit symbol coding (N)

53.91±16.22

59.91±16.25

2.30

0.02*

SF-36

Physical functioning

74.55±28.32

84.55±16.35

0.91

0.36

Physical role functioning

40.91±30.15

79.55±33.20

2.54

0.01*

Bodily pain

68.64±32.32

83.64±12.45

1.54

0.12

General health

49.91±27.88

66.27±22.63

1.84

0.07

Vitality

60.91±32.16

87.27±14.89

2.16

0.03*

Social role functioning

59.91±31.74

80.73±21.93

2.21

0.03*

Emotional role functioning

51.45±43.17

69.64±37.97

1.37

0.17

Mental health

60.73±27.18

77.09±17.17

2.10

0.04*

Statistics from Wilcoxon signed-rank test.

*P<0.05.

6MWT indicates 6-minute walk test; BMI, body mass index; OLB-CE, one-leg balance test with eyes closed; OLB-OE, one-leg balance test with eyes open; SF-36, 36-Item Short Form Health Survey; WHR, waist-hip ratio.


supporting the effects of Baduanjin exercise on physical and psychosocial health domains. Compared with traditional exercise modes, such as aerobic exercise and strengthening exercise that generally involve more physical effort, the Baduanjin exercise could be an effective alternative exercise because it demands less physical effort. Although limited literature is available regarding the effects of Baduanjin on balance function, processing speed, and quality of life in people with SMI, the positive changes on these outcomes were consistent with the results of previous research on Qigong in other clinical groups such as cardiac patients34 and cancer survivors.10

Poor study retention has been recognized as a problem in health promotion programs for people with SMI.20 How to maintain participation in physical activity after completion of a structured exercise group is a major challenge in the health promotion field. Poor adherence to exercise may contribute to decrements in the health benefits achieved after a structured exercise program. One of the strengths of this study was the use of mobile technology (ie, SMS reminders) to encourage participants with SMI to continue participation in physical activity. It has been suggested that such an SMS strategy could be a low cost and successful way to deliver information to change health-related

behaviors in less wealthy and less healthy populations.35 Moreover, more research is needed in the emerging field of applying mobile technology in mental health.23,25

The results of our study support the feasibility, acceptability, and potential efficacy of SMS-based strategies to promote participation in physical activity in clinical populations with SMI. Although the participation rate showed a gradual decline during the follow-up period, participants still attained 80% of targeted exercise time (90 min) with SMS reminders. Interestingly, participants practiced Baduanjin at home >3 days per week, which was more frequent than the prescribed number of days. Although the frequency of participation was somewhat better on the days participants received the SMS reminders than on those when they did not (70% vs. 61%), it should be noted that participants still performed exercise on most of the days without text reminders. In our study, the participants received the SMS reminders on random days, which might have helped maintain their attention to the prompt throughout the study and resulted in them exercising more frequently on average than the prescribed number of days. However, the average duration of the daily practice sessions was <20 minutes, which was shorter than the goal amount. Low motivation is a core feature of psychiatric disorders. An SMS strategy might be useful in promoting exercise behavior in people with SMI; however, more strategies are needed to increase the duration of each practice. The higher frequency with which the picture-based educational brochure and SMS reminders were used compared with the training CD and DVD might reflect the level of accessibility and convenience; lack of media players at home might be a barrier to use of training CDs/ DVDs.

Limitations of this study included the small sample size and the lack of control group. Lack of a control group during the follow-up period makes it difficult to determine the actual impact of the SMS reminders on participation rates. The results should also be interpreted with caution given the short-term (8 wk) study design. Use of the selfreport daily record sheet might also have led to an overestimation of the participation rate during the follow-up period; however, the researchers reviewed the sheets and confirmed details with participants to reduce bias. In addition, >80% of the participants were diagnosed with schizophrenia, which might reduce the generalizability of the study results to a broader mentally ill population. Furthermore, individuals included in the study were those who were participating in a community-based psychiatric rehabilitation center, owned a personal mobile phone, and were able to read text messages. Future research is needed with a larger sample and a wider diversity of participants (eg, outpatients and those who are unfamiliar with mobile phone SMS usage) to generalize the concept of applying communication technology in rehabilitation and health promotion for people with SMI.

CONCLUSIONS

A Baduanjin exercise group session may have the potential to contribute to the physical and

TABLE 3. Acceptability and Frequency of Use of Home-based Exercise Support Materials (N=10)

Acceptability of Materials

Agreement [n (%)]

Disagree

Neutral

Agree

The following support could help me doing Baduanjin at home

Prior 8 wk Baduanjin exercise group

1(10)

1 (10)

8 (80)

Picture-based educational brochure

1(10)

1 (10)

8 (80)

Soundtrack CD

2 (20)

1 (10)

7 (70)

Video CD/DVD

2 (20)

1 (10)

7 (70)

SMS reminders

1 (10)

1 (10)

8 (80)

In general, all of the support material could help me doing Baduanjin at home

1 (10)

1 (10)

8 (80)

SMS reminders did not bother me

1 (10)

2 (20)

7 (70)

Marking daily record sheets did not bother me

1 (10)

1 (10)

8 (80)

Frequency of use

Seldom

Sometimes

Often

Used picture-based educational brochure

2 (20)

2 (20)

6 (60)

Played soundtrack CD

7 (70)

2 (20)

1 (10)

Played video CD/DVD

8 (80)

0

2 (20)

Read SMS reminders

1(10)

1 (10)

8 (80)


psychological health of people with SMI. In this study, participants were able to maintain partial adherence to home-based exercise. An SMS strategy could be a feasible and successful way to encourage individuals with SMI to participate in physical activity. Future studies that enroll larger samples and include a control group are needed to con firm the findings.

REFERENCES

Psychiatry Research 292 (2020) 113346


Contents lists available at ScienceDirect

Psychiatry Research

journal homepage: www.elsevier.com/locate/psychres

Short communication

Clinical outcomes from the texting for relapse prevention (T4RP) in schizophrenia and schizoaffective disorder study



Bernadette A. Cullena,b,*, Katrina Rodriguezb, William W. Eatonb, Ramin Mojtabaia,b, Tara Von Machc, Michele L. Ybarrab,d

a Department of Psychiatry and Behavioral Sciences, The Johns Hopkins Medical Institutions, Baltimore, Maryland, USA

b Department of Mental Health, The Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

c Ann & Robert H. Lurie Children's Hospital of Chicago, USA

d Center for Innovative Public Health Research, San Clemente, California, USA

ABSTRACT

This 6 month randomized control trial investigated whether a novel text-messaging program impacted targeted clinical outcomes in patients with schizophrenia and schizoaffective disorder (SAD). Forty patients were enrolled and completed baseline, 3-month and 6-month assessments. The intervention group received daily symptom check-in text messages, plus, a medication reminder or, inspirational quote text. The control group had treatment as usual. At 6 months the Positive and Negative Syndrome Scale mean positive score was significantly lower and injectable medication compliance was significantly higher in the intervention group. Recovery scores were significantly higher at 3 months. Results suggest that this program may benefit individuals with schizophrenia/SAD who use text messaging. Further investigation in a larger sample appears warranted.

Introduction

Schizophrenia and schizoaffective disorder (SAD) are neurodeve-lopmental disorders with significant public health impact. Schizophrenia is the fifth leading cause of years lived with disability among men and sixth among women and is responsible for 1% of the global burden of disease (Eaton, 2019). For people with schizophrenia the one-year reported relapse rate ranges between 17-41% (Robinson et al., 1999) and the 5-year relapse rate may be as high as 82% (Robinson et al., 1999). SAD also has a high relapse rate (Angst et al., 1980). Recurrent relapses correlate with poorer treatment response (Emsley et al., 2013) and poorer quality of life (Briggs et al., 2008), accelerate social impairment (Hogarty et al., 1991) and increase the likelihood of having residual symptoms (Shepherd et al., 1989).

Programs that target relapse prevention (Herz and Lamberti, 1995) can potentially change the disorder trajectory for those with schizophrenia/SAD. These programs, however, are expensive to implement and so remain unavailable to many patients (Eisner et al., 2013). The increasing use of text messaging provides opportunities to develop relapse prevention programs that are readily accessible to people with schizophrenia/SAD. Nationally, up to 81.4% of people with a serious mental illness report owning a cell phone (Firth et al., 2016), and text messaging has been reported as the second most common use of their phones (Ben-Zeev et al., 2013a).

There has been growing interest in the potential role that technology could play in the detection and management of early symptoms of relapse in schizophrenia. Spaniel (Spaniel et al., 2008) reported a year-long trial of an automated text messaging program where physicians initiated a pre-determined medication increase when early symptoms of relapse were identified. This intervention led to a 60% reduction in the number of hospitalizations among participants.

Granholm and colleagues (Granholm et al., 2012) developed Mobile Assessment and Treatment for Schizophrenia, a text-messaging program that targeted medication adherence, social interaction and auditory hallucinations. Medication adherence improved among those who completed the program and the severity of hallucinations decreased. However, those with more severe negative symptoms, lower functioning, and lower premorbid IQ had a high dropout rate. Participants also found the study-provided phones difficult to navigate.

A study in China found that sending educational and medicationreminder text messages daily to individuals with schizophrenia and a family member/support person increased medication compliance and reduced relapses and hospitalizations. The program, however, relied on the family member/support person being actively involved and neither program content nor the frequency/timing of the text messages could be customized (Xu et al., 2019). Almost 70% of participants required training on cell phone use.

Building on past research, we developed the Texting for Relapse Prevention Program (T4RP) for individuals with schizophrenia/SAD (Ybarra ML, 2019). This is a text-messaging program developed through a collaborative approach involving focus groups with patients, providers, and administrators, which targets early warning symptoms of relapse in those with schizophrenia/SAD. The program is designed for use by those who have a cell phone and use text messaging. At the start of the program the patient, along with their provider, identifies their top five early warning symptoms of relapse. The T4RP program then sends daily text messages to the patient querying them on one of these five symptoms. The patient responds to symptom check-ins by texting “yes”, “no”, or a numerical response depending on the particular question. If the symptom is present, they receive a follow up text message with a rotation of coping skill suggestions specific to that symptom. If they are asymptomatic, they receive a supportive text message. Additionally, the patient receives, on a daily basis, either a medication-related text message (adherence/coping with side effects) or an inspirational quotation text message. When the patient endorses the presence of a symptom, they are asked about it again on subsequent days, and provided with coping skills targeting the symptom until it resolves. They also continue to receive the daily check-in text message about another of their 5 early warning symptoms. If the patient endorses their preset, individualized threshold of symptoms, the program alerts their provider through either email or text message. The provider contacts the patient within 24 hours to decide on further management. This could include adjusting medication, referral to case management, or problem solving on social stressors.

* Corresponding author at: Meyer 144, 600 Nth Wolfe St., Baltimore, MD 21287, USA.

E-mail address: bcullen@jhmi.edu (B.A. Cullen). https://doi.org/10.1016/j.psychres.2020.113346

Received 7 February 2020; Received in revised form 26 July 2020; Accepted 28 July 2020

Available online 29 July 2020

0165-1781/ © 2020 Elsevier B.V. All rights reserved.


We posit that T4RP will reduce psychiatric morbidity and institutional rates and promote recovery by promoting self-management of early warning symptoms, medication adherence and by facilitating improved patient-provider communication. Here we report on the impact of the pilot T4RP program on number of hospitalizations, experience of symptoms, medication adherence, empowerment and communication with providers.

This study was a randomized controlled trial (RCT) of the T4RP program involving patients with schizophrenia/SAD attending a hospital-based community psychiatry program. Eligibility criteria included: a chart diagnosis of schizophrenia/SAD; English speaking; capacity to consent; owning and texting on a cellphone; agreeing to retain the same number for the duration of the study; at their personal baseline as assessed by their provider; agreeing to attend the clinic for the duration of the study; and having their provider agree to participate in the study. All providers in the program were eligible to participate and all consented to participate. The study was reviewed and approved by Advarra IRB and the Johns Hopkins School of Medicine IRB.

Over the course of 3 months, all patients attending the clinic with a chart diagnosis of schizophrenia/SAD were assessed by their provider for study eligibility. Potential participants were referred to the study coordinator for confirmation of eligibility, including assessment for capacity to consent (Dunn and Jeste, 2001). Eligible patients were then consented, completed baseline assessments, and were randomly assigned to the control or intervention arm of the study. Participants were randomized by a computer algorithm at a 2:1 ratio for the intervention and control group to maximize the amount of pilot data collected for those in the intervention. Initially, patients were randomized after they consented to take part in the study. After five people were enrolled this methodology was changed to randomize after the baseline survey was completed.

Patients in the intervention met with their providers to select their 5 common symptoms of relapse, their preferred daily timeframe to receive texts, the symptom threshold to alert their provider and their emergency contact information. The provider entered this information into the T4RP online clinical platform. Intervention patients started receiving text messages the day after they were enrolled in the study and continued receiving daily text messages for 6 months. The protocol for the delivery of text messages followed that outlined above. At enrollment, intervention patients were instructed that should they experience a crisis at any point they could text the word “crisis” and they would receive information on the clinic emergency protocol.

Both intervention and control patients continued to attend their regular clinic appointments.

Clinical assessments were completed at baseline, 3 months, and 6 months. The study coordinator was trained to complete any clinician-administered scales and entered this data directly into the online study platform. In the presence of the study coordinator, patients completed self-report scales directly into the online system.

Primary clinical outcomes included symptoms of relapse, recovery, and institutionalization. Symptoms of relapse were measured using three scales. The Positive and Negative Syndrome Scale (PANSS) (Kay et al., 1987) has three subscales that measure positive symptoms of schizophrenia (alpha = 0.55), negative symptoms of schizophrenia (alpha = 0.60), and general psychopathology (alpha = 0.70). Studies frequently use changes in scores on the positive subscale as an indicator of relapse (Gleeson et al., 2010; Wang et al., 2018). The Montgomery-Asberg Depression Rating Scale (MADRS) measures changes in depressive symptoms during treatment. Items are rated on a scale from 0 to 6 (alpha = 0.79) (Davidson et al., 1986). The Young Mania Rating Scale (YMRS) assesses the presence and severity of manic symptoms (alpha = 0.57) (Young et al., 1978).

Recovery was measured using the Recovery Assessment Scale Revised (RAS-R), a 24-item self-report scale with 5 subscales that measures personal recovery in those with serious mental illness (Corrigan et al., 2004). This scale is sensitive to change over time (Salzer and Brusilovskiy, 2014).

Information on hospitalizations, emergency room (ED) visits and intensive outpatient program (IOP) referrals the year before baseline and for the past 3 months at 3- and 6-month follow-ups was obtained from providers on all study patients.

Posited targeted mechanisms included medication adherence, patient empowerment, and improved patient-provider communication. Medication adherence was measured using the Brief Adherence Rating Scale (BARS), a 4-item scale designed for use in community settings with individuals with schizophrenia/SAD. We compared those with 90% or better medication adherence to those with less than 90% adherence. Depot injection medication adherence rates were measured by comparing the number of injections the patient received to the number they should have received based on the prescribed frequency of delivery of the depot medication. Those with 90% or better adherence were compared to those with less than 90% adherence.

Patient empowerment was measured with the Boston University Empowerment Scale (BUES) short version, a 25-item self-report scale that measures empowerment among those using mental health services (alpha = 0.94) (Rogers et al., 2010).

To measure patient-provider communication, participants were asked how strongly they agreed or disagreed with five statements that described their relationship with their provider (e.g., “My provider knows who I want involved in my care if my symptoms get worse”). Responses were summed to create a communication “score” (alpha = 0.90).

Missing data on the BUES scale due to a participant declining to answer some items was replaced with scale mean values. This affected 7 people at baseline and 8 people at 6-month follow-up. One additional

person was not included in the baseline BUES scale because there was an internet connectivity issue and their data for this scale were lost. None of the other scales had missing data.

Differences between the intervention and control participants were tested using chi-square tests for dichotomous measures (e.g., medication adherence) or t-tests for continuous measures (e.g., depressive symptomatology). The sample size for this study was determined by the project's main aims, which were to determine the program's feasibility and acceptability. As such, it was not powered to detect statistical differences but rather to generate estimates to inform power estimates for a larger trial. Given the exploratory nature of this study, multivariable analyses were not conducted. Also given the pilot nature of the work, a permissive p-value of 0.20 was used to guide differences between the experimental arms (Stallard, 2011).

Twelve hundred and twenty patients were assessed for eligibility, 42 of whom were consented and randomized. The main reason for ineligibility was not using text messaging. One person was randomized to the control group before completing the baseline survey and was censored from analysis as they withdrew when assigned to the control group. Another person was censored post-hoc when their medical record indicated they did not have a diagnosis of schizophrenia/SAD. Of those who completed the baseline survey (n = 40), 95% (n = 38) completed the 3-month follow-up and 92% (n = 37) completed the 6-month follow-up surveys. Two people actively withdrew from the study, one for unknown reasons following their baseline assessment and one after their 3-month assessment due to experiencing an exacerbation of their symptoms. One person remained in the study but, due to scheduling issues, was unable to complete either the 3- or 6-month assessments.

Participants were, on average, 49 years old, 2 in 5 were female, most identified as Black/African American, most were single and two in three had a diagnosis of schizophrenia (Table 1). The intervention and control groups were balanced on demographic characteristics.

Symptoms of Relapse: Positive PANSS scores were significantly lower at baseline (M; 16.5 vs. 13.3, p = 0.03) and at 6-month follow-up (M: 14.3 vs. 11.4, p = 0.03) for the intervention group. Symptoms of depression and mania were not significantly different between the two groups across the 6-month study (Table 2).

Recovery: Several indicators of recovery were significantly better for those in the intervention group at 3-month follow-up, including personal confidence and hope (M: 25.6 vs.28.8 p = 0.05), goal and success orientation (M: 19.8 vs 21.7 p = 0.03), reliance on others (M: 15.1 vs 16.8, p=0.03), and the total score (M: 82.4 vs 90.4, p=0.04). Significantly higher scores did not persist through 6 months (Table 2).

Institutionalization. No significant differences in hospitalizations, emergency room visits, or IOP referrals were noted between the two groups at either follow-up period (Table 2).

Posited mechanisms of change. Improved oral medication adherence was suggested at both 3 and 6 months for those in the intervention group (69% vs 89%, p = 0.11). Greater rates of adherence to injectable medications were noted in the intervention group at both 6 months (31% vs 100%, p = 0.02) and baseline (0% vs 57%, p = 0.09). Patient empowerment scores were higher in the intervention group over time, particularly at 6-months (Table 2).

Using text messaging to identify early symptoms of relapse for people who have schizophrenia/SAD has the potential to be a user-

Table 1

Demographic characteristics of pilot RCT participants.

Demographic characteristics           Control Intervention

(n=12)      (n=28)

M (SD)       M (SD)             p-value

Age                                   50.1 (14.2)    48.1 (13.2)          0.68

% (n)

% (n)

Female

41.7 (5)

42.9 (12)

0.94

Race

0.52

White

16.7 (2)

3.4 (1)

Black/ African American

83.3 (10)

85.7 (24)

All other

0 (0)

10.8 (3)

Hispanic

0 (0)

3.6 (1)

0.51

Ethnicity

Marital status

0.36

Married

8.3 (1)

7.1 (2)

Divorced

25.0 (3)

17.9 (5)

Single

58.3 (7)

71.4 (20)

Widowed

8.3 (1)

3.6 (1)

Employment status

0.17

Full time

0 (0)

10.7 (3)

Part time

0 (0)

7.1 (2)

Not employed

100.0 (12)

82.2 (23)

Insurance type

0.64

Medicare

66.7 (8)

60.7 (17)

Medicaid

33.3 (4)

32.1 (9)

Private insurance

0 (0)

7.1 (2)

Psychiatric Diagnosis

0.57

Schizoaffective disorder

33.3 (4)

42.9 (12)

Schizophrenia

66.7 (8)

57.1 (16)

Participated in intervention

16.7 (2)

35.7 (10)

0.23

development activity

friendly and cost-effective intervention. To our knowledge, T4RP is one of the first text messaging-based programs that was developed with significant input from patients and providers. It also is novel because the program tailors the daily check-in messages and coping skills to the early warning signs of the individual. Interactions with the program are simple and use skills (texting) that the target population already has. While the development of self-management apps (Ben-Zeev et al., 2013b; Eisner et al., 2019; Kidd et al., 2019; Palmier-Claus et al., 2013a, Palmier-Claus et al., 2013b) provides another tool to help patients manage their symptoms, T4RP may be more accessible to this patient population than an app which requires access to a smart phone.

Findings from this study are encouraging. Those in the intervention had fewer positive symptoms than those in the control group at 6 months post-enrollment and their recovery scores were significantly higher at 3 months. Although there was a significant difference in the number of positive symptoms between the two groups at baseline, a consistent decline in symptoms was observed in the intervention but not the control group. It may be that the coping skill messages worked as intended, and empowered patients to enact strategies to address their symptoms. The trend towards higher empowerment scores in the intervention group at both 3 and 6 months is also encouraging.

Medication adherence is important in preventing symptom relapse (Leucht et al., 2012) and impacts mortality rates among those with schizophrenia (Cullen et al., 2013). The significant improvement in adherence to injectable medication and trend towards improved oral medication adherence suggests that this may be a mechanism for change. Individuals in the intervention group received, on average, 3 messages per week that either encouraged them to take their medication or provided simple solutions for common side effects. Previous studies have had success improving medication adherence with daily reminder text messages (Granholm et al., 2012; Montes et al., 2012Xu et al., 2019). Findings from the current study suggests that patient

Table 2

Clinical outcomes for the pilot Texting 4 Relapse RCT.

Patient characteristics

Baseline (n = 40)

3-month outcomes (n=38)

6-month outcomes (n=37)

Control (n=13) M (SD)

Intervention (n=28)

M (SD)

p-value

Control (n=12) M (SD)

Intervention (n=26)

M (SD)

p-value

Control (n=12) M (SD)

Intervention (n=25)

M (SD)

p-value

Clinical indicator

Symptoms of relapse

PANSS

Positive

16.5 (4.5)

13.3 (4.0)

0.03

14.3 (4.5)

12.2 (3.4)

0.14

14.3 (3.7)

11.4 (3.5)

0.03

Negative

12.3 (3.7)

13.6 (4.7)

0.37

12.8 (4.0)

13.8 (5.2)

0.55

11.9 (4.6)

14.4 (5.8)

0.21

General

30.3 (8.1)

26.7 (6.5)

0.15

24.8 (7.0)

24.2 (5.2)

0.79

24.5 (6.9)

22.2 (5.3)

0.27

Total

59.0 (13.2)

53.6 (11.2)

0.20

51.8 (10.8)

50.2 (10.0)

0.67

50.7 (10.5)

48.0 (10.3)

0.46

Montgomery-Asberg Depression Rating Scale

9.6 (9.1)

8.1 (7.6)

0.60

8.7 (9.3)

5.5 (7.2)

0.26

6.2 (7.8)

5.4 (7.3)

0.78

Young Mania Rating Scale

5.1 (3.7)

4.1 (2.8)

0.34

2.9 (1.0)

3.0 (0.5)

0.93

3.3 (0.7)

2.6 (0.5)

0.41

Recovery

Recovery Assessment Scale

Personal confidence and

25.8 (4.0)

27.3 (5.2)

0.38

25.6 (4.0)

28.8 (4.9)

0.05

25.9 (4.1)

26.3 (6.5)

0.86

hope

Willingness to ask for help

12.0 (2.1)

12.6 (2.2)

0.42

12.2 (2.0)

12.8 (1.8)

0.34

12.1 (1.7)

12.1 (2.6)

0.96

Goal and success orientation

19.8 (2.6)

21.0 (3.6)

0.29

19.8 (2.0)

21.7 (2.5)

0.03

20.1 (2.3)

20.8 (4.0)

0.55

Reliance on others

14.8 (2.4)

16.3 (3.2)

0.15

15.1 (1.4)

16.8 (2.5)

0.03

14.5 (2.3)

15.9 (3.5)

0.22

Not dominated by

9.3 (2.6)

8.6 (2.7)

0.51

9.8 (2.3)

10.2 (2.4)

0.56

9.3 (2.1)

9.9 (2.5)

0.49

symptoms

Total recovery score

81.7 (10.7)

85.9 (12.1)

0.30

82.4(9.3)

90.4 (11.3)

0.04

81.9 (10.3)

85.0 (16.8)

0.56

Institutionalization

% (n)

% (n)

% (n)

% (n)

% (n)

% (n)

Hospitalizations (1+)

15 (2)

11 (3)

0.58

0 (0)

11 (3)

0.21

0 (0)

7.7 (2)

0.31

Psychiatric ER visits

15 (2)

7 (2)

0.33

0 (0)

0 (0)

NC

0 (0)

4 (1)

0.47

IOP referrals

0 (0)

4 (1)

0.49

0 (0)

0 (0)

NC

0 (0)

4 (1)

0.47

Posited targeted mechanisms

Oral medication adherence (>=90%)*

75% (9)

75% (21)

NC

85%(11)

96% (27)

0.18

69% (9)

89% (25)

0.11

Injectable medication adherence

0% (0)

57% (4)

0.09

67% (2)

88% (7)

0.43

33% (1)

100% (6)

0.02

(>=90%)**

Patient Empowerment (BUES)

59.4

(10.1)***

55.3 (11.6)

0.03

69.6 (5.3)

72.5 (7.1)

0.22

71.0 (4.7)

74.7 (8.1)

0.15

Improving communication between

20.3 (4.0)

20.8 (3.7)

0.70

20.1(3.2)

21.3 (.4)

0.21

19.8 (0.3)

19.8 (4.7)

0.99

patients and providers

*33 people were on oral medications at baseline, 34 at 3 months, and 36 at 6 months. **10 people were on injectable medications at baseline, 11 at 3 months, and 9 at 6 months.

***1 control participant's responses were lost due to an internet problem.


education may also influence medication adherence.

Contrary to our hypotheses, hospitalizations, emergency room visits, and IOP referral rates were similar for the intervention and control groups. Perhaps this is because a text messaging program is insufficient, on its own, to affect these large clinical outcomes. It also is possible that were the sample size larger, differences in these rates would be easier to detect.

Patient-provider communication scores were relatively stable for both the groups during the course of the study. This may reflect the baseline level of engagement of patients with providers in this particular clinical setting. Testing the intervention in different types of clinical settings may help further illuminate the impact that T4RP has in this regard.

This study had some limitations. The study sample includes only those people who had a cell phone and used text messaging. In the city where the study was conducted, Medicaid-facilitated phone ownership included unlimited text messaging, which facilitated the inclusion of low-income people in the study. In other settings where this type of subsidy is not available, T4RP may not be appropriate. Also, T4RP requires experience with text messaging so it is not suitable for everyone. This study was a pilot study designed to assess feasibility and acceptability. As such, the sample size was based upon feasibility and was not sufficient to detect statistical differences between the groups. Additionally, the internal consistency for the positive PANSS subscale and the Young Mania Rating Scale in the current study were sub-optimal. This perhaps made it more difficult to detect a signal.

Results from the pilot study of the T4RP program are promising, particularly in terms of the posited mechanism for change, improved medication adherence; and the clinical outcome, reduced positive symptoms. Future research that includes multiple sites to determine whether the program impacts patient-provider communication in nonacademic settings, and institutionalization in a larger clinical sample appears warranted.

CRediT authorship contribution statement

Bernadette A. Cullen: Conceptualization, Methodology, Writing -original draft, Funding acquisition. Katrina Rodriguez: Investigation, Data curation, Writing - review & editing. William W. Eaton: Conceptualization, Writing - review & editing. Ramin Mojtabai: Conceptualization, Writing - review & editing. Tara Von Mach: Investigation, Writing - review & editing. Michele L. Ybarra: Conceptualization, Methodology, Formal analysis, Writing - review & editing, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2020.113346.

References

Angst, J., Felder, W., Lohmeyer, B., 1980. Course of schizoaffective psychoses: results of a followup study. Schizophr. Bull. 6 (4), 579-585.

Ben-Zeev, D., Davis, K.E., Kaiser, S., Krzsos, I., Drake, R.E., 2013a. Mobile technologies among people with serious mental illness: opportunities for future services. Adm. Policy Mental Health 40 (4), 340-343.

Ben-Zeev, D., Kaiser, S.M., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., 2013b. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatry Rehabil. J. 36 (4), 289-296.

Briggs, A., Wild, D., Lees, M., Reaney, M., Dursun, S., Parry, D., Mukherjee, J., 2008. Impact of schizophrenia and schizophrenia treatment-related adverse events on quality of life: direct utility elicitation. Health Qual. Life Outcomes 6, 105.

Corrigan, P.W., Salzer, M., Ralph, R.O., Sangster, Y., Keck, L., 2004. Examining the factor structure of the recovery assessment scale. Schizophr. Bull. 30 (4), 1035-1041.

Cullen, B.A., McGinty, E.E., Zhang, Y., Dosreis, S.C., Steinwachs, D.M., Guallar, E., Daumit, G.L., 2013. Guideline-concordant antipsychotic use and mortality in schizophrenia. Schizophr. Bull. 39 (5), 1159-1168.

Davidson, J., Turnbull, C.D., Strickland, R., Miller, R., Graves, K., 1986. The Montgomery-Asberg depression scale: reliability and validity. ACTA Psychiatry Scand. 73 (5), 544-548.

Dunn, L.B., Jeste, D.V., 2001. Enhancing informed consent for research and treatment. Neuropsychopharmacology 24 (6), 595-607.

Eaton, W.W., Bienvenu, O.J., Nestadt, G., Volk, H.E., Anthony, J.C, 2019. The Burden of Mental Disorders, Public Mental Health, Second. Oxford University Press, New York.

Eisner, E., Bucci, S., Berry, N., Emsley, R., Barrowclough, C., Drake, R.J., 2019. Feasibility of using a smartphone app to assess early signs, basic symptoms and psychotic symptoms over six months: a preliminary report. Schizophr. Res. 208, 105-113. https://doi.org/10.1016/j.schres.2019.1004.1003. Epub 2019 Apr 1019.

Eisner, E., Drake, R., Barrowclough, C., 2013. Assessing early signs of relapse in psychosis: review and future directions. Clin. Psychol. Rev. 33 (5), 637-653.

Emsley, R., Oosthuizen, P., Koen, L., Niehaus, D., Martinez, L., 2013. Comparison of treatment response in second-episode versus first-episode schizophrenia. J. Clin. Psychopharmacol. 33 (1), 80-83.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2016. Mobile phone ownership and endorsement of "mHealth" among people with psychosis: a metaanalysis of cross-sectional studies. Schizophr. Bull. 42 (2), 448-455.

Gleeson, J.F., Alvarez-Jimenez, M., Cotton, S.M., Parker, A.G., Hetrick, S., 2010. A systematic review of relapse measurement in randomized controlled trials of relapse prevention in first-episode psychosis. Schizophr. Res. 119 (1-3), 79-88.

Granholm, E., Ben-Zeev, D., Link, P.C., Bradshaw, K.R., Holden, J.L., 2012. Mobile assessment and treatment for schizophrenia (MATS): a pilot trial of an interactive textmessaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull. 38 (3), 414-425.

Herz, M.I., Lamberti, J.S., 1995. Prodromal symptoms and relapse prevention in schizophrenia. Schizophr. Bull. 21 (4), 541-551.

Hogarty, G.E., Anderson, C.M., Reiss, D.J., Kornblith, S.J., Greenwald, D.P., Ulrich, R.F., Carter, M., 1991. Family psychoeducation, social skills training, and maintenance chemotherapy in the aftercare treatment of schizophrenia. II. Two-year effects of a controlled study on relapse and adjustment. Environmental-personal Indicators in the course of schizophrenia (EPICS) research group. Arch. Gen. Psychiatry 48 (4), 340-347.

Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13 (2), 261-276.

Kidd, S.A., Feldcamp, L., Adler, A., Kaleis, L., Wang, W., Vichnevetski, K., McKenzie, K., Voineskos, A., 2019. Feasibility and outcomes of a multi-function mobile health approach for the schizophrenia spectrum: App4Independence (A4i). PLoS One 14 (7), e0219491 0219410.0211371/journal.pone.0219491. eCollection 0212019.

Leucht, S., Tardy, M., Komossa, K., Heres, S., Kissling, W., Salanti, G., Davis, J.M., 2012. Antipsychotic drugs versus placebo for relapse prevention in schizophrenia: a systematic review and meta-analysis. Lancet 379 (9831), 2063-2071.

Montes, J.M., Medina, E., Gomez-Beneyto, M., Maurino, J., 2012. A short message service (SMS)-based strategy for enhancing adherence to antipsychotic medication in schizophrenia. Psychiatry Res. 200 (2-3), 89-95.

Palmier-Claus, J.E., Rogers, A., Ainsworth, J., Machin, M., Barrowclough, C., Laverty, L., Barkus, E., Kapur, S., Wykes, T., Lewis, S.W., 2013a. Integrating mobile-phone based assessment for psychosis into people's everyday lives and clinical care: a qualitative study. BMC Psychiatry 13, 34.

Palmier-Claus, J.E., Rogers, A., Ainsworth, J., Machin, M., Barrowclough, C., Laverty, L., Barkus, E., Kapur, S., Wykes, T., Lewis, S.W., 2013b. Integrating mobile-phone based assessment for psychosis into people's everyday lives and clinical care: a qualitative study. BMC Psychiatry 13, 34. https://doi.org/10.1186/1471-1244X-1113-1134.

Robinson, D., Woerner, M.G., Alvir, J.M., Bilder, R., Goldman, R., Geisler, S., Koreen, A., Sheitman, B., Chakos, M., Mayerhoff, D., Lieberman, J.A., 1999. Predictors of relapse following response from a first episode of schizophrenia or schizoaffective disorder. Arch. Gen. Psychiatry 56 (3), 241-247.

Rogers, E.S., Ralph, R.O., Salzer, M.S., 2010. Validating the empowerment scale with a multisite sample of consumers of mental health services. Psychiatry Serv. 61 (9), 933-936.

Salzer, M.S., Brusilovskiy, E., 2014. Advancing recovery science: reliability and validity properties of the recovery assessment scale. Psychiatry. Serv. 65 (4), 442-453.

Shepherd, M., Watt, D., Falloon, I., Smeeton, N., 1989. The natural history of schizophrenia: a five-year follow-up study of outcome and prediction in a representative sample of schizophrenics. Psychol. Med. Monogr. Supp. 15, 1-46.

Spaniel, F., Vohlidka, P., Hrdlicka, J., Kozeny, J., Novak, T., Motlova, L., Cermak, J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res. 98 (1-3), 312-317.

Stallard, N., 2011. Optimal sample sizes for phase II clinical trials and pilot studies. Statistics in Medicine. Wiley Online Library.

Wang, D., Gopal, S., Baker, S., Narayan, V.A., 2018. Trajectories and changes in individual items of positive and negative syndrome scale among schizophrenia patients prior to impending relapse. NPJ Schizophr. 4 (1), 10.

Xu, D.R., Xiao, S., He, H., Caine, E.D., Gloyd, S., Simoni, J., Hughes, J.P., Nie, J., Lin, M., He, W., Yuan, Y., Gong, W., 2019. Lay health supporters aided by mobile text messaging to improve adherence, symptoms, and functioning among people with schizophrenia in a resource-poor community in rural China (LEAN): a randomized controlled trial. PLoS Med. 16 (4), e1002785 1002710.1001371/journal.pmed.1002785. eCollection 1002019 Apr.

Ybarra, ML, R.K., Madison H, Mojtabai, R, Cullen, BA, 2019. Developing texting for relapse prevention: a salient mhealth program for people with schizophrenia and schizoaffective disoreder. J. Nerv. Mental Disord. press.

Young, R.C., Biggs, J.T., Ziegler, V.E., Meyer, D.A., 1978. A rating scale for mania: reliability, validity and sensitivity. Br. J. Psychiatry 133, 429-435.

Single-Session Mobile-Augmented Intervention in Serious Mental Illness: A Three-Arm Randomized Controlled Trial


Colin A. Depp*,1,2, Dimitri Perivoliotis1,2, Jason Holden1, Jennifer Dorr2, and Eric L. Granholm1,2

'Department of Psychiatry, University of California, San Diego, San Diego, CA; 2Psychology Department, VA San Diego Healthcare System, San Diego, CA

*To whom correspondence should be addressed; 9500 Gilman Drive, La Jolla, CA, US; tel: 858-822-4251, fax: 858-534-5475, e-mail: cdepp@ucsd.edu


Psychosocial interventions for serious mental illness are resource intensive and poorly accessible. Brief interventions (eg, single session) that are augmented by follow-on automated mobile health intervention may expand treatment access. This was a randomized single-blind controlled trial with 255 individuals diagnosed with schizophrenia or bipolar disorder. Participants were randomized to one of three conditions: CBT2go, which combined one individual session of cognitive behavioral therapy with automated thought chal-lenging/adaptive behavior delivered through mobile devices; Self-Monitoring (SM), which combined single-session illness psychoeducation with self-monitoring of symptoms; and treatment-as-usual (TAU). Participants were assessed at baseline, 6 weeks (midpoint), 12 weeks (posttreatment), and 24 weeks (follow-up) with our primary outcome global psychopathology (Brief Psychiatric Rating Scale-expanded version [BPRS-24]), and secondary outcomes community functioning (Specific Level of Function; SLOF) and defeatist performance beliefs (DPBs). We also collected data on adverse events. Outcome analyses on the primary outcome, BPRS Total score, indicated a significant time (0-24 wk) by group interaction with significant but modest improvement comparing two active conditions (CBT2go and SM) relative to TAU. Effects of CBT2go were not different from SM. There was a significant time x group interaction with better SLOF scores in CBT2go across 24 weeks, but not in SM. There were no time-by-group effects on DPBs. DPBs decreased in the CBT2go condition but not in SM. These results indicated that single intervention augmented by mobile intervention was feasible and associated with small yet sustained effects on global psychopathology and, when inclusive of CBT, community function compared with usual care.

Key words: bipolar disorder/schizophrenia/psychotherapy/ technology/Internet-based treatments/depression/ecological momentary assessment

Introduction

Limited access to evidence-based interventions for serious mental illnesses (SMI; schizophrenia, bipolar disorder) has resulted in efforts to create brief or low-intensity interventions to expand reach of treatments1 including through mobile technology.2,3 Rates of mobile device usage and ownership in SMI are increasing,4 and mobile health interventions are of interest to the population.5,6 Moreover, despite concerns raised over the quality of extant clinical trials, reviews have generally indicated positive impact on symptoms.7,8 Although one meta-analysis suggested differential impact across design features,8 a few studies have experimentally tested variations in mobile intervention content or whether treatment effects are sustained after acute phases of intervention. To address these gaps, we conducted a randomized controlled trial (RCT) contrasting two types of single-session interventions augmented by mobile health: cognitive behavioral therapy (CBT) vs illness education with self-monitoring.

In a previous trial in bipolar disorder, we found greater impact on depressive symptoms when a 4-session psychoeducation intervention was augmented by mobile intervention compared with psychoeducation alone, immediately posttreatment.9 Although that trial supported mobile augmentation, there are a variety of design considerations for mobile health interventions10,11 that can impact the user, time for personnel delivering the intervention, and the cost and complexity of deployment. Moreover, mounting evidence indicates mobile applications are associated with a steep decline in engagement following initiation.12 Our prior trial found attenuation of effect of mobile augmentation 12 weeks after active encouragement of device use.9 Finally, mechanism of change in mobile intervention is poorly understood, such as whether self-monitoring by itself may affect change (as in other behaviors13) or if therapeutic elements that draw from evidence-based interventions such as CBT are impactful beyond selfmonitoring. A recent proof-of-concept trial evaluated CBT vs self-monitoring content in schizophrenia in early psychosis and found positive impact of CBT compared with self-monitoring on psychotic symptoms.14 That trial, although highly promising, did not include a treatment-as-usual (TAU) condition and was sized to be focused on feasibility and acceptability rather than outcomes.

© The Author(s) 2018. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com


Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


Few studies have evaluated the impact of design features on mechanisms. Defeatist beliefs are an intervention target for CBT in psychosis and may be useful in disentangling the impact of CBT vs self-monitoring.15,16 A recent meta-analysis13 of the relationship between defeatist attitudes found consistent associations with negative symptoms and functioning but with small effect sizes, because multiple factors contribute to poor functioning. The relationship, however, is sufficiently robust such that we found improvement in defeatist attitudes in cognitive-behavioral social skills training (CBSST) significantly mediated improvements in negative symptoms and functioning, and participants with more severe defeatist attitudes showed significantly greater improvement in functioning in CBSST.17-19

We developed a single-session in-person intervention called CBT2go augmented by mobile interactions for SMI. Building from prior work,9,20 this intervention integrated in-person training with mobile frequent assessment of self-management targets (ie, current mood symptoms or voices, socialization, and medication adherence), related defeatist beliefs (eg, lack of perceived control of symptoms), and adaptive beliefs and behaviors. We developed a Self-Monitoring (SM) alternative, in which inperson content concerned psychoeducation and resources pertinent SMI, and mobile interaction only involved selfmonitoring without CBT elements. CBT2go and SM were contrasted to a TAU condition. The primary outcome was change in global psychopathologic symptoms and the secondary outcomes were functioning and defeatist performance beliefs (DPBs). We hypothesized that the two mobile interventions would result in significant decreases in symptoms and improvements in functioning compared with TAU and that the CBT2go intervention would result in greater improvements than the SM condition.

Methods

Participants and Recruitment

Participants were outpatients diagnosed with schizophrenia, schizoaffective disorder, or bipolar I disorder. The target accrual was 255 and participants were recruited between December 2013 and February 2017. Recruitment was through residential facilities and clinics affiliated with the San Diego County Mental Health System, self-help support groups, and online advertisements. The frequency of referral source for the entire sample, in descending order, was residential facility or congregate living (60.5%), referral from other study (21.5%), outpatient clinic (14.9%), referral from friends (2.6%), and advertisement (0.4%). This referral source distribution did not significantly differ by group assignment (CBT2go, SM, TAU; x2(8) = 7.3, P = .533). To be eligible, participants needed to be (1) aged 18 years and older; (2) outpatients prescribed stable psychotropic medication regimens for the prior 3 months; (3) rated greater than 3 (1 = not present to 7 = extremely) on at least one of the Brief Psychiatric Rating Scale (BPRS) items 3 (Depression), 7 (Elevated Mood), 10 (Hallucinations), or 17 (Emotional Withdrawal). Note, these items were selected due to their relevance to the targeted domains in the Ecological Momentary Assessment protocol and positive, negative and mood symptom clusters in the target population. (4) Free of visual or manual dexterity disabilities that would preclude operation of a touch screen device. We excluded participants who (1) were intoxicated at the time of interview; (2) were psychiatrically hospitalized in the prior month; (3) had participated in CBT within the past 5 years; (4) had a diagnosis of dementia, seizure disorder, intellectual disability, or experienced a past head injury with a loss of consciousness greater than 20 minutes; or (5) were actively participating in another clinical trial.

This study was approved by the University of California, San Diego (UCSD) Human Subjects Protections Program. All participants provided written informed consent confirmed by passing the UCSD Brief Assessment of Decisional Capacity.21 Diagnosis was determined with the Mini International Neuropsychiatric Interview for Diagnostic and Statistical Manual, Fourth Edition11, chart review, and consensus meetings with the principal investigator. Participants were compensated $50 for time spent in each assessment (they were not compensated for intervention). The Clinicaltrials.gov number was NCT02035202. Prior publication with these data have concerned baseline characteristics.23

Randomization. Randomization with three-group 1:1:1 ratio was completed by an independent statistician via computerized random number generator. Participants were assigned to one of the following: (1) CBT2go, (2) SM, or (3) TAU. Raters were masked to study assignment. Masking was preserved by (a) separating case discussions with therapist from raters, (b) counseling participants not to reveal randomization status, and (c) replacing raters for all subsequent assessments in cases of breaking blind.

Intervention Conditions

Treatment-as-Usual (TAU). Participants only completed assessments. As per eligibility criteria, all participants were participating in outpatient psychiatric follow-up and required to be prescribed medications for their mental health diagnosis at the time of study entry. Participants were linked with care in case of crisis during research assessments.

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


CBT2go In-Person Session. Participants met with a therapist in the community for a single 90-minute session. The treatment manual contained an introduction to the program and to the cognitive behavioral model, and then modules on mood symptoms, voices, socialization, and medications. We provided participants the option of completing the mood (depression) or voices sections (or both on alternating days) based on the symptom domain that participants believed most impacted their functioning. Each module provided psychoeducation about the topic and queried participants about their experience and current strategies for self-management. Participants then were presented with common defeatist beliefs that corresponded to the topic (eg, for depression, “I have no control over my symptoms;” for socialization, “Others won’t like me”). The therapist and participant then collaboratively selected more balanced beliefs (eg, “Joe likes to hang out with me at the clubhouse”) and a behavioral strategy linked to the belief (eg, ask to Joe to do someone fun). Participants were encouraged to personalize the cognitive and behavioral strategies to increase relevance. As such, a variety of “if-then” scripts were created linking maladaptive beliefs to sets of corresponding adaptive beliefs and behaviors. Participants were also asked about strategies for wellness, and personalized encouraging statements were presented linked to endorsement of low levels of symptoms or positive adherence.

Self-Monitoring (SM) In-Person Session. As with CBT2go, SM was a single 90-minute session that included psychoeducation about the diagnosis, causes, symptoms, and treatments for mental illness, and the importance of self-monitoring symptoms; unique manuals were developed for bipolar disorder and schizophrenia that provided content specific to that diagnosis. Participants were encouraged to ask questions about their diagnosis and treatment, and were provided a list of resources (eg, support groups). The manual was designed not to include any content regarding maladaptive beliefs or the cognitive-behavioral model.

Therapist and Fidelity. The same therapist (JD), a master’s-level clinician, conducted all in-person appointments for both active conditions and was not masked to condition. Sessions were audiotaped and the therapist completed a fidelity rating each session. Audiotapes of sessions were reviewed by two independent fidelity raters who were masked to condition; they were also asked to guess which condition was delivered in order to monitor potential crossover effects. An 8-item fidelity rating form was adapted from the Cognitive Therapy Rating Scale for Psychosis24 with a score range of 0-16, with a parallel version for SM content. The therapist also provided a report of the proportion of CBT2go or SM material covered in the session, with a percentage between 0% and 100%. Supervision occurred in weekly meetings.

Mobile Interactions

Mobile Device Technical Description. Participants assigned to either CBT2go or SM were provided with an Internet-enabled smartphone or could elect to use their own phones. The rate of use of participant own phones during the course of the study was approximately 10% in both CBT2go and SM conditions. A web-based program called Mobile Online Behavioral Intervention Technology (MOBIT) delivered interactive surveys to the device that contained elements personalized from the individual session. At each survey epoch, users received an invitation to complete a “survey” at a randomly scheduled time within 3 daily blocks of time (morning, afternoon, and evenings). Interactions were triggered via SMS that automatically opened a web application. Responses were recorded by use of a touch screen interface with categorical responses, and all data resided on a server housed at UCSD. At the conclusion of the in-person session, participants were trained on how to operate the device and responding to alarms. Participants also received a written manual describing the operation of the device.

CBT2go Interactive Content. CBT2go algorithms contained assessment and intervention content. The goal for the mobile component was not meant to be a stand-alone intervention, but to augment in-person content by providing real-time thought challenging intervention outside of the clinic setting individualized to the specific symptoms or defeatist beliefs they endorsed at the time. Different from homework in traditional in-person therapy, which is self-initiated, the mobile device prompted participants to engage in cognitive restructuring. Mood or Voices algorithms were delivered during the morning survey, Socialization in the afternoon, and Medication Adherence in the evening. The first question pertained to symptom severity/frequency, socialization, and medication adherence and the second to presence of one of 3-4 current maladaptive beliefs corresponding to that domain (eg, “I have little control over my voices”). Participants could also select “none of these” in relation to the maladaptive beliefs. The intervention content branched from each of the maladaptive beliefs to offer a potential alternative or adaptive belief, personalized by the individual in the in-person session, accompanied by adaptive behaviors. Participants then rated their intention to engage in the activity on one of three levels (eg, “I probably/might/ will do it”) and received feedback based on this statement (eg, if “I might do it,” then “Go ahead and give it a try”). These algorithms were developed to be completed in 1-2 minutes.

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


Self-Monitoring Content. The SM intervention contained the same questions about the frequency/severity of symptoms, socialization, and medication adherence, without any of the intervention content. In both SM and CBT2go participants could on demand review symptoms through a graphical application.

Follow-Up Telephone Contacts. In both the CBT2go and SM conditions, the study therapist contacted participants by phone 4 times (wk 1, 4, 7, and 10) to provide reminders of upcoming assessment appointments, encourage adherence to the surveys, and troubleshoot. CBT2go participants could also elect to change elements of the personalized responses at that time. At the end of 12 weeks, participants returned study devices and could elect to re-route surveys to their own devices if desired.

Measures

Assessment Schedule and Rater Training. Participants were assessed at baseline and 6-, 12-, and 24-week follow-up. Raters were trained to administer the BPRS via videotaped gold standard cases and attained at least 0.80 interrater reliability.

Demographics and Diagnosis. All participants were assessed at baseline for basic sociodemographic information, diagnosis and treatment history, and current participation in treatment, including medications.

MATRICS Consensus Cognitive Battery (MCCB). The MCCB used to assess global cognitive performance for sample characterization at the baseline visit.25 We administered tests only in the domains of Verbal Learning, Reasoning/Problem Solving, Working Memory, and Processing Speed; domain scores were normed for age and education and then combined for a composite T-score.

Primary Outcome

Symptoms. The Brief Psychiatric Rating Scale-expanded version (BPRS-24) was used to measure psy-chopathologic symptoms26 including anxiety, depression, mania, delusions/hallucinations, unusual behavior, and negative symptoms.

Secondary Outcomes

Functioning. The Specific Level of Functioning Scale (SLOF)27 measures 4 domains: Interpersonal Functioning (eg, social participation); Everyday Activities (eg, instrumental activities of daily living); Work Skills (eg, ability to complete tasks), and Social Acceptability (eg, managing conflict). Consistent the Validation of Everyday Real-World Outcomes (VALERO) study,28 a best-estimate approach was used, in which interviewers combined information from interview, participant self-report, and informants. Informants were professionals with high levels of contact (eg residential facility managers). We did not include the Personal Care or Physical Functioning subscales given that they are frequently at ceiling in community-dwelling outpatients.29 We did not administer the SLOF at the 6-week assessment to minimize burden on informants.

Defeatist Performance Beliefs. The Defeatist Performance Attitude Scale (DPAS) is a 15-item self-report subscale using items from the Dysfunctional Attitude Scale.30 The DPAS indexes endorsement of defeatist attitudes about one’s ability to perform goal-directed tasks (eg, “People will probably think less of me if I make mistakes”).

Adverse Events. The study was monitored by a Data and Safety Monitoring Board (DSMB) annually. The timing, duration, and aftermath of psychiatric and nonpsychiatric hospitalizations were collected.

Statistical Analyses. We evaluated the association between attrition and baseline characteristics. Generalized linear mixed models were performed with subject as a random effect and fixed effects for condition (CBT2go, SM, and TAU), time (0, 6, 12, and 24 wk), and for the primary outcome (BPRS) baseline score due to a significant difference between randomized and participants who completed the intervention. Time was entered as a continuous variable and an autoregressive (AR1) covariance structure was used. Planned contrasts were performed in contrasting the combined and independent comparisons of the active conditions to TAU (CBT2go, SM vs TAU) and between the two active conditions (CBT2go vs SM). Effect sizes comparing treatment arms were calculated at 6, 12, and 24 weeks as Cohen’s d,31 dividing differences between estimated means by the pooled baseline raw standard deviation,32 as well as estimated change from baseline to 24 week follow-up in the same manner. We calculated number needed to treat (NNT) by comparing 25% response rates on the primary outcome at 24 weeks. Finally, we explored moderation by diagnosis status (bipolar disorder vs schizophrenia/schizoaffective disorder) on our primary outcome. We evaluated the global association between survey completion and follow-up phone call completion by Pearson correlation between these adherence indicators and estimated 24-week BPRS Total change scores. The a level was set to .05.

Results

Sample Ascertainment and Characteristics. The Consolidated Standards of Reporting Trials (CONSORT) figure displays flow through the trial (figure 1). The recruitment target was met and retention in the trial was reasonable, with an overall retention rate at 24 weeks of

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018



Allocation

Allocated to CBT2go (n=85)

Declined to continue post-randomization (n=B)

Received allocated intervention (n=77)

Allocated to Self-Monitoring (n=85)

Declined to continue post-randomization (n=14)

Received allocated intervention (n=69)

Allocated to Treatment as Usual (n=85)

Declined to continue post-randomization (n=2)

-

k.

Follow-Up

Lost to follow up at 6 weeks (n=1)

Lost to follow-up at 12 weeks (n=1)

Lost to follow-up at 24 weeks (n=3)

All drop out participants were lost to contact

Lost to follow up at 6 weeks (n=2)

Lost to follow-up at 12 weeks (n=1)

Lost to follow-up at 24 weeks (n=5)

All drop out participants were lost to contact

Lost to follow up at 6 weeks (n=7)

Lost to follow-up at 12 weeks (n=0)

Lost to follow-up at 24 weeks (r?=3)

All droo out Darticioants were lost to contact

Analysis

Analysed (n=77)

Analysed (n=69)

Analysed (n=83)

Figure 1. Consort diagram.


84.7%. The 24-week retention rate was lower in the SM condition (77%) compared to CBT2go (91%) and TAU (88%), x2(2) = 6.5, P = .037). Participants who dropped out had higher BPRS scores at baseline than those who completed all 4 assessments (dropout = 46.3 (12.2) vs 42.0 (10.4), F(1,250) = 5.5, P = .020), but were not different on any other sample characteristic. Sensitivity analyses indicated that the association between baseline BPRS score and dropout was only significant in the SM condition (F(1,82) = 7.3, P = .009). On average (table 1), the sample was middle-aged, exhibiting cognitive ability 1 standard deviation below average and experiencing a mild level of severity of psychopathology.33

In-Person Session and Follow-Up Contact Fidelity and Adherence. Fidelity rating scales indicated a high level of fidelity to both the SM and CBT2go, and scores were not different between conditions (CBT2go: 16.0, SD = 0.0, SM: 15.9, SD = 0.22; t(144) = 0.9, P = .346). Similarly, participant comprehension did not differ between conditions (CBT2go: 15.0, SD = 0.3, SM: 14.9, SD = 0.3, t(144) =0.2, P = .822). Masked raters were able to differentiate 100% of tapes into the correct condition and audio review fidelity ratings were high and consistent with therapist ratings (M = 15.7 of 16, SD = 1.0). Therapist ratings of proportion of manualized content covered within the session averaged 94.5% (SD = 15.9) for the SM condition and 93.6% (SD = 16.8) in the CBT2go condition, which was not significantly different (Mann-Whitney U-test z = 0.2, P = .813). Of the 4 scheduled follow-up phone calls, the average proportion completed was 56% (SD = 0.32). The average duration was 8.7 minutes in the CBT2go condition (SD = 7.0) and 8.0 minutes in SM (SD = 5.8); the rate of call completion or duration did not differ between conditions (P = .912 and .612 for CBT2go and TAU, respectively).

In terms of adherence to the mobile device interactions, mean adherence (% of surveys responded during the monitoring period) aggregated across modules was similar between the CBT2go and SM conditions (CBT2go: M = 68.7%, SD = 27.4; SM: M = 66.2%, SD = 29.9, t(358) = 3.0, P = .413). Within the individual modules, rates of adherence were higher for the evening survey on medication adherence in the CBT2go condition (t(117) = 2.2, P = .025), but there were no other differences. Neither the rate of survey completion (r = .073, P = .616) nor phone call completion (r = -.079, P = .560) were associated with estimated change in BPRS Total scores. Similarly, rates of survey completion (r = .089, P = .527) and follow-up phone calls (r = -.061, P = .685)

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


Table 1. Baseline Characteristics (n = 229)

CBT2go (n = 77)

SM (n = 69)

Mean (SD) or %

TAU (n = 83)

Mean (SD) or %

Mean (SD) or %

Age

51.2 (11.5)

49.4 (11.1)

48.1 (11.7)

Sex (% female)

45.8

53.3

49.2

Ethnicity

White/non-Hispanic

46.5%

41.0%

43.2%

African American

16.3%

24.1%

17.3%

Asian

5.8%

9.6%

6.2%

Latino/Hispanic

27.9%

24.1%

33.3%

More than one ethnicity

4%

0%

0%

Education (y)

12.9 (2.1)

12.2 (2.3)

12.1 (2.2)

Marital status (% married)

9.8

14.6

4.9

Living situation

Independent living, in community

47.7%

59.2%

49.8%

Residential facility

50.0%

39.8%

46.3%

Homeless

2.3%

1.0%

4.9%

Diagnosis

Schizophrenia/Schizoaffective

71.4%

78.3%

75.9%

Bipolar disorder

28.6%

21.7%

24.1%

Age of first onset

23.2 (9.1)

23.2 (9.1)

21.3 (8.1)

MATRICS total (T-score)

38.6 (8.8)

38.1 (10.3)

38.3 (9.2)

Medications prescribed

Antipsychotic

84.4%

87.0%

82.1%

Mood stabilizer

26.0%

17.4%

19.3%

Antidepressant

58.3%

46.4%

61.4%

Note: SM, Self-Monitoring; TAU, Treatment-as-usual.


were not significantly associated with BPRS change in the SM condition.

Primary Outcome. Planned comparisons for BPRS Total were the following: (1) two active conditions and TAU (CBT2go/SM vs TAU), (2) the two active conditions (CBT2go vs SM), and (3) each active conditions and TAU (table 2). There was a significant time x visit interaction in comparing the two active conditions to TAU. There were no significant time x group interactions when contrasting the CBT2go vs SM conditions, or each of the individual conditions to TAU. Treatment effects were small (d = 0.23 at 2 wk for CBT2go and d = 0.22 for SM). The average estimated improvement was significant in the CBT2go condition (estimated BPRS improvement =3.53 points, SE = 1.02, t = 3.4, P < .001, pre-post change Cohen’s d = 0.36) and in the SM condition (estimated BPRS improvement = 3.10 points, SE = 1.01, t = 2.8, P = .005, d = 0.26). Finally, in regard to treatment response (25% improvement in BPRS Total scores), compared to TAU (9.6%), response rates were 21.1% in the CBT2go condition (NNT = 8.7) and 15.6% in the SM condition (NNT = 15.6).

Secondary Outcomes. There was a significant group x time effect for community functioning (SLOF) favoring CBT2go vs TAU (table 3). Treatment effects were smallmedium at 24 weeks for CBT2go (d = 0.36). Scores in TAU condition were in the direction of worsening over time, whereas the active conditions were greater by week 24. Page 6 of 11

The estimated average change was not significant in the CBT2go condition (estimated SLOF improvement = 2.1 points, SE = 1.8, t = 1.1 P = .254, pre-post d = 0.14), SM condition (estimated SLOF improvement = 0.3 points, SE = 1.9, t = 0.2, P = .864, pre-post d < 0.01), or TAU condition (estimated SLOF worsening = -3.0 points, SE = 1.7, t = 1.7, P = .087, pre-post d = 0.19).

There were no time x group interactions for DPBs (table 4). However, estimated change indicated significant improvement in the CBT2go condition (estimated DPB improvement = 4.73 points, SE = 1.7, t = 2.8, P = .005, d = 0.25) but not in the SM condition (estimated DPB improvement = 1.82, SE = 1.8, t = 1.0, P = .311, pre-post d = 0.10).

Adverse Events. There were 31 adverse events, experienced by 21 different participants in the study (12 in SM, 10 in CBT2go, and 9 in TAU). These events were all hospitalizations (12 medical and 19 psychiatric). Two participants were dropped from the study as a result of being placed in long-term care facilities due to medical conditions; the remainder returned to their residence and resumed participation. These events were reported to the DSMB and were determined unlikely to be related to the study interventions and consistent population risk.

Exploratory Outcome Analysis by Diagnosis. In an exploratory analysis, we evaluated whether diagnosis (schizophrenia/schizoaffective disorder vs bipolar

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


Table 2. Brief Psychiatric Rating Scale Total Score Across the Study Period by Condition

BPRS Total

CBT2go

SM

TAU

Active Conditions vs TAU

CBT2go vs SM

CBT2go vs TAU

SM vs TAU

M (SD)

n

M (SD)

n

M (SD)

n

Adj. Effect

Size (</), P value

Adj. Effect

Size (</), P value

Adj. Effect

Size (</), P value

Adj. Effect

Size (</), Rvalue

Baseline

43.2 (9.7)

76

41.6(11.5)

69

42.0(10.9)

83

N/A

N/A

N/A

N/A

6 Wk

41.0(11.1)

75

39.5 (7.7)

67

41.0(11.3)

76

0.10, P = 03.51

0.02, P = .803

0.08, P= .385

0.10, P = .357

12 Wk

39.7(11.5)

74

38.6(9.1)

66

41.0(11.4)

76

0.18, P = .Ill

<0.01, P = .685

0.11, P= .113

0.13, P = .259

24 Wk

39.7(11.9)

71

37.6(10.1)

61

41.1 (11.4)

73

0.22, P = .028

<0.01, P = .972

0.23, P = .053

0.21, P = .068

Significance tests

Time x Grp: F

Time x Grp:

Time x Grp:

Time x Grp:

for difference in

(1,862) = 3.9,

F(l,405) = 1.3,

F(l,599) = 3.9,

F(l,566) = 3.9,

slopes

P = .048

P = .262

P = .071

F = .129

Table 3. Specific Level of Function (SLOF) Scores Across the Study Period by Condition

SLOF Total

CBT2go

SM

TAU

Active Conditions vs TAU

CBT2go vs SM

CBT2go vs TAU

SM vs TAU

M (SD)

n

M (SD)

n

M (SD)      n

Adj. Effect

Size (</), P value

Adj. Effect

Size (<7), P value

Adj. Effect

Size (<7), P value

Adj. Effect

Size (<7), P value

Baseline

126.9 (15.2)

73

128.4(15.5)

69

126.9(16.1)    79

N/A

N/A

N/A

N/A

12 Wk

129.1 (15.8)

74

127.8(16.1)

66

123.4(15.8)    75

0.18, P = . 106

0.12, P = .353

0.30, P= .063

0.24, P = .383

24 Wk

130.0(16.9)

71

128.7(16.3)

61

123.3(18.0)    72

O.27F = .O15

0.11, P = .402

0.36, P = .011

0.33, P = . 107

Significance tests

Time x Grp: F

Time x Grp: F

Time x Grp: F

Time x Grp:

for difference in

(1,630) = 3.3,

(1,554) = 0.1,

(1,437) =4.0

F(l,414) = 1.0,

slopes

P = .068

P = .705

P = .046

P = .325

8103 Jaqopo bO uo jasn Ajejqn A;isjsaiup |eoips|/\| 6unisqoe>| Aq 6S8tA |,g/g£|,Aqs/|nqqos/£60101/!op/pe.i)sqe-a|O!pe-aoueApe/u!)a||nqe!uajqdoz!qos/woodnoo!wapeoe//:sdpq luojj papeo|UMoa







Table 4. Dysfunctional Performance Belief Scores Across the Study Period by Condition








disorder) moderated primary outcome. This analysis should be interpreted with caution due to the imbalance in sample sizes between diagnostic groups. The bipolar disorder and schizophrenia subgroups did not differ on age (7(227) = 1.8, P = .067), gender (x2(1) = 1.5, P = .206), or ethnicity (x2(5) = 9.5, P = .108). Patients with bipolar disorder had significantly lower (less severe) scores on the BPRS Total compared to patients with schizophrenia (39.3 [SD = 6.8] vs 43.2 [SD = 11.2], t(1) = 2.4, P = .017], and baseline BPRS Total was incorporated into the model. We found a 3-way interaction between group x time x and diagnosis (F(6,226) = 2.7, P = .011). Inspection of the effect sizes compared to TAU indicated that 12-week visit effects were large for CBT2go in the bipolar group (d = 1.24; SM vs TAU: d = 0.20), and negligible for the schizophrenia (CBT2go vs TAU: d = 0.01; SM vs TAU: d = 0.11). However, at 24 weeks, these effects were small-moderate for CBT2go in the schizophrenia/ schizoaffective group (d = 0.33; SM condition vs TAU: d = 0.26), whereas they were minimal in the bipolar group (CBT2go vs TAU: d = -0.21; SM vs TAU: d = 0.07).

Conclusions

This was among the largest RCTs of mobile health in SMI to date, and it was unique in that two different single-session mobile-augmented interventions were compared. Participants who received interventions experienced greater improvement in global psychopathology than TAU. On balance, the magnitude of this impact was small. in addition, community functioning improved more in the CBT2go vs TAU condition, and the effect size was small-medium. DPBs also significantly improved in the CBT2go condition but not in the SM or TAU conditions. Each intervention was generally well tolerated, although pretreatment dropout was higher in the SM condition, and participants with more severe BPRS scores at baseline were more likely to drop out of that condition. It is unclear why the SM intervention was associated with greater dropout, but because this occurred prior to intervention, it is possible that SM may be less appealing to patients experiencing more severe symptoms. Finally, the pattern of results indicated sustainment of improvement at 24-week follow-up. Overall, single-session interventions augmented by mobile health intervention resulted in modest yet sustained positive impact on global psychopathology, with more selective positive impact on attitudes and community functioning when incorporating elements of CBT.

Study strengths included the relatively large sample size, 3-group design, and systematic collection of fidelity and adverse event data noted to be lacking in some prior RCTs of mobile interventions. However, there were several limitations. The sample was primarily middle-aged with an average duration of illness of approximately 20 years, and so these results may not generalize to first

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


onset populations. The sample was treated and had on average a mild level of symptoms and so the results may not generalize to untreated or more acutely ill populations. The low level of symptoms also restricted range of possible improvement. Adaptations may be needed to reach more severely ill populations. Our study design did not enable us to disentangle which of the in-person session or mobile health augmentation was associated with observed improvements, and we cannot rule out that primary driver of impact was the in-person session and/or follow-up calls. We did evaluate whether the rate of participation in survey completion or follow-up calls was associated with global change in our primary outcome, which may have provided some indication which aspects of the intervention were associated with the changes observed. However, we did not find global associations in either the CBT2go or SM conditions, and so the “active ingredients” of the interventions described here are still unclear. Nonetheless, it may be that a deeper investigation of trajectories or patterns of adherence to the components of the interventions could shed light on how participant engagement throughout the trial impacted outcomes. Furthermore, both of our interventions involved active clinician delivery and follow-up telephone calls, which is a model that is less scalable than completely automated interventions. Moreover, the same therapist delivered study intervention in both conditions, and whereas masked ratings of fidelity were nearly identical across conditions, we fully cannot rule out potential bias; moreover, the use of a single therapist may limit generalizability as therapist training and competence was standardized across patients. The rate of follow-up call completion (56%) was lower than expected, which was due to difficulty scheduling remote calls with participants.

There was mixed evidence for the incremental benefit of CBT content over and above self-monitoring, psychoeducation, and novelty of using a mobile device.34 None of the head-to-head comparisons between CBT2go and SM were significant, yet CBT2go was superior to TAU on community functioning, whereas SM was not. Our trial results were consistent with a recent proof-of-concept study indicating relative benefit of CBT vs self-monitor-ing.14 That trial was somewhat different in that app use was incentivized and the population focus on early psychosis; therefore, in addition to the distinction between CBT and self-monitoring content, future research should examine the extent to which sampling and implementation strategies such as incentivization impact outcome.

Much more needs to be learned about the active ingredients of smartphone-delivered interventions. The limited understanding about which specific mechanisms of mobile interventions produce impact is evident as well in other disorders such as depression8 and is derivative of the long-standing debate regarding the active ingredients of evidence-based psychotherapies in general. Active ingredients may vary within an illness application depending on which outcome is being targeted and the sustainability of these effects beyond supervised participation in the mobile intervention. For example, our study suggests that the incorporation of CBT elements may be more necessary, relative to self-monitoring alone, for addressing community function than for symptom management. Therefore, future work should evaluate active ingredients across a range of outcomes and durability of changes after cessation of active periods of coaching. CBT2go was associated with significant within-group improvement on the defeatist attitudes, consistent with the general model for CBT in psychosis, whereas SM was not. Given that DPBs have been linked to asociality, amotivation, and poor functioning in schizophrenia,15 it is possible that the greater impact of CBT2go on functioning was due to the impact of CBT elements on defeatist beliefs. Because frequent data can be collected on hypothesized treatment mechanisms and outcomes, mobile interventions could be particularly useful in delineating which intervention mechanisms impact outcomes by evaluating mediation at the day-to-day level. Future design improvements could strengthen the impact of CBT-based mobile interventions, which could include user-driven and on-demand interaction, adaptive and scaffolded interaction based on an individual’s prior responses over time, and devicetriggered (rather than scheduled and frequently missed in this study) interactions with providers. In a traditional RCT, isolating the impact of these features is implausible given the large number of potential features. Emerging adaptive research designs35 could be used to evaluate the impact of these design elements, perhaps on their immediate impact on mechanisms like defeatist beliefs.

Finally, there was preliminary evidence of differences between diagnoses (schizophrenia vs bipolar disorder) in the pattern of treatment response; this variation by diagnosis should be interpreted with caution due to unequal sample sizes and diagnostic variation at baseline in severity which was adjusted for statistically. In bipolar disorder, CBT2go effects were large at the conclusion of the active phase of treatment but dissipated at follow-up, consistent with our prior study8; however, participants with schizophrenia experienced improvements relative to TAU at follow-up that were undetectable at the end of active monitoring. Pending replication, interventions may need to be further tailored in content or perhaps duration of contact to different diagnoses.

In terms of clinical implications, an important consideration is where CBT2go and similar other mobile-augmented interventions might fit in the care continuum. The magnitude of effects we observed was attenuated compared to brief (~10 session)1 or intensive36 (20+ session) interventions, yet this is weighed against the total clinician time of 2 hours per participant. Moreover, the NNT for reduction in overall symptomology was 8.7 for CBT2go, which compares favorably to outcomes of brief 6-session CBT delivered by nurses.37 Rather than a replacement for standard evidence-based therapies for serious mental illness, the potential clinical application of CBT2go (and other single-session interventions) may likely be in settings in which there is little or no access to evidence-based psychotherapies for SMI, given that the intervention maybe easier to scale than more prolonged treatments that have to date been poorly disseminated. In addition, within settings that do offer more intensive interventions, CBT2go may be a potential solution to enhance operational efficiency, such as a first stage in stepped care, or as a treatment alternative among patients with low levels of symptoms to conserve access to higher intensity care for more severely ill people. Further enhancements, such as by delivering in-person sessions through videoconferencing or online, may further enhance scalability such as to patients with limited geographic access.

Downloaded from https://academic.oup.com/schizophreniabulletin/advance-article-abstract/doi/10.1093/schbul/sby135/5114359 by Kaohsiung Medical University Library user on 04 October 2018


Funding

National Institute of Mental Health (MH100417 to C.A. D.).

Conflict of Interests

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry. 2013;35:332-338.


Psychological Medicine, Page 1 of 14. © Cambridge University Press 2017 doi:10.1017/S0033291717001982

ORIGINAL ARTICLE


Effectiveness of a low support, remotely accessible, cognitive remediation training programme for chronic psychosis: cognitive, functional and cortical outcomes from a single blind randomised controlled trial

J. F. Meaney6, B. Fitzmaurice2, B. Hallahan3, C. McDonald3, T. Wykes4, A. Corvin2 and I. H. Robertson5

4Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, England

5 Trinity College Institute of Neuroscience, Trinity College Dublin, Ireland, Trinity College Dublin, Ireland

6National Centre for Advanced Medical Imaging (CAMI), St. James's Hospital/School of Medicine, Trinity College Dublin, Dublin, Ireland

Background. Cognitive remediation (CR) training has emerged as a promising approach to improving cognitive deficits in schizophrenia and related psychosis. The limited availability of psychological services for psychosis is a major barrier to accessing this intervention however. This study investigated the effectiveness of a low support, remotely accessible, computerised working memory (WM) training programme in patients with psychosis.

Methods. Ninety patients were enrolled into a single blind randomised controlled trial of CR. Effectiveness of the intervention was assessed in terms of neuropsychological performance, social and occupational function, and functional MRI 2 weeks post-intervention, with neuropsychological and social function again assessed 3-6 months post-treatment.

Results. Patients who completed the intervention showed significant gains in both neuropsychological function (measured using both untrained WM and episodic task performance, and a measure of performance IQ), and social function at both 2-week follow-up and 3-6-month follow-up timepoints. Furthermore, patients who completed MRI scanning showed improved resting state functional connectivity relative to patients in the placebo condition.

Conclusions. CR training has already been shown to improve cognitive and social function in patient with psychosis. This study demonstrates that, at least for some chronic but stable outpatients, a low support treatment was associated with gains that were comparable with those reported for CR delivered entirely on a 1:1 basis. We conclude that CR has potential to be delivered even in services in which psychological supports for patients with psychosis are limited.

Received 19 December 2016; Revised 19 June 2017; Accepted 20 June 2017

Key words: Cognition, cognitive remediation, MRI social function, schizophrenia.

Introduction

Cognitive deficits make a significant contribution to disability in many psychotic disorders (Green et al. 2000; Wykes & Reeder, 2005). Often predating the emergence of clinical symptoms, these deficits persist throughout the illness, despite fluctuations in clinical symptom severity. Because current antipsychotic medications do not adequately treat these deficits (Green, 1996; Fett et al. 2011), behaviour-based therapies designed to remediate cognitive deficits, an approach known as cognitive remediation training (CR), have become a significant focus of research.

CR targets difficulties with cognitive skills such as attention, memory, problem-solving, informationprocessing-speed, organization, planning and social cognition. Despite significant differences between training programmes in both methods and administration, a meta-analysis by Wykes et al. (2011) based on over 2000 participants found consistent evidence of cognitive gains associated with CR, yielding an average effect size of 0.45 across the range of interventions considered. Importantly, these benefits are not confined to cognition, but were also associated with benefits to social and occupational functioning (Wykes et al. 2011).

Several questions about CR remain, including the cost-effectiveness of the various approaches taken and their potential for integration into standard clinical care (Patel et al. 2010). Even if the cost of CR compares favourably to currently used pharmacotherapy, the number of therapist hours involved are typically substantial, representing a potential challenge given the generally limited availability of psychosocial treatments for psychosis. Efforts to address this issue have included group-based training (Lilienthal et al. 2013) and, recently, computer-based training using software that allows task difficulty to be automatically varied according to changes in patients' performance. While having the potential to limit the need for 1:1 support for each session, an important question for such 'e-health' initiatives is to determine patients' capacity to carry out such 'remote' training and how much training support is required to adequately facilitate participation.

In a preliminary 'proof of concept' study, we recently investigated whether patients benefitted from one such remote training programme in terms of improved memory function. This training programme is specifically focused on training working memory (WM) - the ability to ability to maintain and manipulate information over a period of seconds - following a hypothesis that WM training benefits cognitive function more generally, and may potentially achieve these gains more efficiently than programmes with a more general focus. In support of this hypothesis, training programmes that exclusively focused on WM in non-schizophrenia populations have been associated with a transfer of benefits to other cognitive functions, including attention, problem-solving and fluid intelligence (Jaeggi et al. 2008; Jaeggi et al. 2010; Lilienthal et al. 2013; Rudebeck et al. 2012; Salminen et al. 2012; Kundu et al. 2013). In psychosis, only a few studies have exclusively targeted WM training, but with promising results (Wexler et al. 2000; Haut et al. 2010; Hubacher et al. 2013; Subramaniam et al. 2014). Of these, the study by Subramanian further reported that WM training-related cortical changes were associated improved occupational functioning. In our proof of concept study, patients who trained remotely and received only 30-60 min of support per week from a therapist showed significant benefits in memory performance compared with patients receiving treatment as usual (Hargreaves et al. 2015).

Improved cognitive performance has repeatedly been associated with changes in cortical structure and function. Reviews of WM-based intervention in particular (Klingberg, 2010; Constantinidis & Klingberg, 2016) suggest that WM training is associated with changes in brain activity in frontal and parietal cortex and basal ganglia, as well as changes in dopamine receptor density. Klingberg further suggests that observed transfer of the training effects to non-trained WM tasks is consistent with the notion of training-induced plasticity in a common neural network for WM. While evidence supporting training-induced changes in cortical activity is widespread in non-psychosis samples (Constantinidis & Klingberg, 2016), evidence of similar training-related changes in schizophrenia is only beginning to be reported (Li et al. 2015).

The aim of this study (registered at ClinicalTrials. gov with ID: NCT01903707) was to investigate the effectiveness of this low support remotely accessed computerised WM training programme in a single blind randomised controlled trial (RCT) of patients with psychosis. Following on our proof of concept study (Hargreaves et al. 2015), we hypothesised that, compared to an active control condition, WM training would result in cognitive improvements in both (untrained) WM task performance, and performance on other memory tasks (e.g. verbal episodic memory). We also hypothesised, based on the studies cited above, that this training, if successful, would result in improved general cognitive function (based on measures of general intelligence), and in improved social and occupational function. We further hypothesised that any improvements observed would be associated with increased neural response during WM task performance using functional MRI (fMRI). Finally, given the evidence that resting state functional connectivity correlates with WM capacity (Stevens et al. 2012), and strengthens the fronto-parietal network following training (Jolles et al. 2013; Takeuchi et al. 2013), we tested the hypothesis that CR training would result in the same functional connectivity in this group also.

Methods

Participants

Following ethical approval for the study, 90 participants were recruited from community health teams from various clinical services in Dublin, Wicklow and Galway, and through the Dublin branch of the National Learning Network, a community-based rehabilitation service. Patients were referred by their local treatment or rehabilitation teams following a series of presentations made about CR by the study team. Diagnosis was confirmed by a trained psychiatric research nurse based on a Structured Clinical Interview for DSM-IV (SCID) and review of all available information - including family or staff report, and chart review. Criteria for inclusion in the study were that participants were aged between 18 and 65 years, had a history of psychosis, were community-based and clinically stable (in the opinion of the treating team), and were engaged in some activity (e.g. part time work, attending a rehabilitation clinic for at least 2 days each week). Exclusion criteria included a history of organic impairment, head injury resulting in loss of consciousness, or drug abuse in the preceding 3 months.

Following study approval from relevant ethics boards, participants were enrolled after giving informed written consent. Participants did not receive compensation for participation in the study, although travel costs related to MRI participation were reimbursed. While enrolled, in addition to CR/Control treatment, all participants continued to receive treatment as usual (clinical review and medication) and no patient (to our knowledge) received other psychological-based treatments while enrolled in the study.

Randomisation procedure for group allocation and blinding

A randomisation table, based on a stratified block sampling procedure based on age (over and under 40 years old) and gender, was created by an independent statistician and administered by the CR therapist following baseline assessment. Only the CR therapist and participants were aware of group allocation and extensive steps were taken to maintain blinding. Communication about the group allocation of participants was not allowed, separate offices were used by the CR therapist and assessors in addition to separate storage and management of data. Similarly, participants were instructed to not divulge their group allocation to the follow-up assessor. At the end of the study, assessors were asked to guess the treatment allocation. The probability of assessors guessing the correct group was approximately 50%, suggesting successful blinding. Trial design and primary outcome measures did not differ from those originally registered.

Figure 1 shows the study consort diagram. Of the 85 patients who participated in either the treatment arm or control arm, data for 55 (64.7%) were available at

CR training

An online CR training programme, that was developed by us and specifically targeted WM, was used in the study (McAvinue et al. 2013; Hargreaves et al. 2015). Prior to commencing training, computer access and training needs (for software usage) of participants were evaluated. If the participants did not have internet access, a laptop and internet dongle were provided for the duration of training, as was any training required with accessing and logging on to the training website (approximately half of those who completed the programme).

The programme, which was web-based, targeted both auditory and visual WM modalities following Baddeley's (2000) model. The programme was an 8-week programme participants were required to complete within a 12-week window and consisted of: (a) psycho-education on the nature of WM, (b) strategybased learning and (c) practice of nine WM focused training exercises that were gradually introduced over a 5-week period, beginning with the easier exercises first. Participants were required to complete 3040 min of training a day, 5 days a week. The nine WM training tasks, which consisted of n-back tasks and classic digit span tasks were designed to be, to at least some extent, ecologically valid by relating training to every-day tasks. To achieve this, a context was presented for each task; for example, on the n-back faces task, the context given was that the participant was at a party and being introduced to various individuals and had to remember the last four faces they saw, etc. Task difficulty level was automatically adjusted in terms of the speed and amount of information presented according to patients' progress in training, based on the criteria of achieving an accuracy 80% on a previous difficulty level. At the end of each session participants were given visual feedback via a graph of time in training and scores obtained, so that they could track their individual progress. Participants met with the study therapist once per week for the 8 weeks of the training for 45 min. The sessions with the therapist following a structured motivational interviewing approach to support training, and focused on: (a) the applicability of training to real-life situations, (b) use of WM strategies in daily activities (e.g. chunking to remember telephone numbers, use of a mental blackboard'), (c) reinforcement of the generalisability of WM training by reviewing how participants had employed the training in daily life in the preceding week and ways of increasing this in the following week, and (d) problem solving any difficulties encountered using the programme. Usage of the programme in terms of minutes and hours of exercises complete was monitored online by the therapist.

Active control condition

The control condition was designed to mirror the amount of time the participants spent with the therapist in the intervention group. As the first randomised trial of this low support training intervention, this approach was taken rather than including a sham training condition to enable us establish the full effects of the WM training intervention. In the first condition


to Participants in the control group met weekly with the same CR therapist as the intervention condition and for the same amount of time (on average 45 min in both groups) and were encouraged to discuss topics of the participants' choice in an open-ended conversation. Topics of conversation spanned from patientsprevious week's events to hobbies and current affairs. Participant symptoms were not directly approached and there was no set agenda for each session apart from a weekly check-in to establish the participantswell-being. Notes pertaining to each session's content were recorded and the therapist aimed to be nondirective and empathetic in approach.

Clinical and neuropsychological assessment

In addition to completing a SCID assessment with either a clinical research nurse or postdoctoral level psychologist, all patients completed the schedule for the assessment of positive symptoms and the schedule for the assessment of negative symptoms (SAPS and SANS; Andreasen, 1984,1989) and gave details regarding age at illness onset, number of previous hospital admissions and current medication dosage, which was converted into chlorpromazine equivalents. After this information was collected, all patients completed the following measures of neuropsychological, social functional and well-being both prior to training (described as baseline data) and within 2 weeks of completing training (described as 2-week posttraining). Neuropsychological measures were also then readministered 3-6 months later.

Primary outcome measures were included as follows

Episodic memory was assessed using the logical memory subtest, immediate and delayed conditions, from the Wechsler Memory Scale, 3rd edition (Wechsler, 1998).

WM; verbal and spatial memory was assessed using letter number sequencing (LNS) from WMS-III and the spatial WM (SWM) from The Cambridge Neuropsychological Test Automated Battery (Robbins et al. 1994).

Secondary outcome measures were included as follows

General cognitive ability was measured using the similarities and matrix reasoning subtests from the Wechsler abbreviated scale of Intelligence (WASI; Wechsler, 1999), which is a brief, reliable measure of cognitive ability routinely used in clinical, educational and research settings. From these two subtests, a measure of full scale IQ was derived based on the published norms available for the two subtest version of the test.

Executive functioning was measured using the CANTAB Stockings of Cambridge task (SOC), the computerised Wisconsin Card Sorting Task (Wisconsin Card Sorting Test® Computer Version 4) (Heaton & Staff, 2003), the STROOP (Stroop, 1935) and the Trail making test (TMT) (Reitan & Wolfson, 1985).

Social cognition was measured using the total scores from the Reading the mind in the eyes task (Baron-Cohen et al. 1997), a measure of mental state decoding, which is an aspect of theory of mind ability.

Social & Functional outcome were measured using three scales: (a) The total score from the Social and Occupational Functioning Assessment Scale (Rybarczyk, 2011), which provides a rating of global social and occupational function independent of clinical symptoms, (b) The total score from the UCSD performance-based skills assessment - brief (Mausbach et al. 2016), which provides a performance-based skills assessment focusing on finances and communication, and (c) the problem-solving subscales from the Independent Living Scales (35), which rates level of independence in managing everyday situations requiring problem-solving skills. Only this subscale of the test was administered to limit the total length of the assessment battery.

Measures of well-being were measured using the Rosenberg self-esteem questionnaire (Rosenberg, 1965), and the World Health Organisation Quality of Life Project (WHO, 1996).

Statistical analysis

Neuropsychological data analysis

All data were processed using Analysis of Covariance (ANCOVA) carried out in SPSS version 22. For both outcome time-points [i.e. (a) 0-2 weeks post-treatment and (b) 3-6-month post-treatment], a series of ANCOVAs were carried out in which performance on each task was entered as the dependent variable (see Table 2 for full list of measures), group (CR v. control) as the independent variable, and baseline performance on the dependent variable was entered as a covariate. Gender and age also served as covariates.

MRI methods

The fMRI n-back sample included 15 CR patients and 15 control patients. The fMRI resting-state sample included 14 CR patients and 15 control patients. All fMRI n-back and resting-state samples were chosen from the larger patient sample, based on whether they consented to participate in MRI scanning and were able to complete MRI scans across two timepoints. One n-back participant and two resting-state participants were excluded due to excessive fMRI signal dropout, leaving a final sample of 29 n-back participants and 27 resting-state participants.

MRI data were acquired using a 3T Philips Achieva MR system using standard methods (described in online Supplemental material). We used a 2-back task to examine WM-dependent changes in neural activity [blood oxygenation level-dependent (BOLD) signal] (38). We also acquired fMRI data during a resting-state scan (rs-fMRI), during which participants kept their fixation on a cross presented on a screen for 7 min.

For both 2-back and resting-state data, preprocessing was performed using Statistical Parametric Mapping (SPM8, v6313) and MATLAB R2014a (v8.3.0.532) including realignment to the mean image, normalisation to MNI (Montreal Neurological Institute) space with a voxel size of 2*2*2 mm3 and smoothing with an 8 mm FWHM (full-width at half-maximum) isotropic Gaussian filter. The Artefact Detection Tools (ART) toolbox was used for artefact detection.

The general linear model was used to perform statistical analysis of 2-back data with a contrast of 2-back > 0-back. Contrast maps were then entered into a flexible factorial model to examine group * time interactions with factors subject (variance set to equal, independence set to yes), group (variance set to unequal, independence set to yes) and time (variance set to equal, independence set to no) (Glascher & Gitelman, 2008).

Rs-fMRI pre-processing and statistical analysis followed the same methods previously used by our group (McCarthy et al. 2013; Mothersill et al. 2016), including functional connectivity analysis using the CONN toolbox (v15; National Institutes of Health Blueprint for Neuroscience Research). Functional connectivity maps generated by this analysis were entered into a flexible factorial model, as with the 2-back analysis. Based on a previous rs-fMRI study from our group (Mothersill et al. 2016), we examined functional

Table 1. Demographic and clinical characteristics of the intervention and control groups

Baseline

CR group (n = 48)

Mean (s.d.)

Control group (n = 42)

Mean (s.d.)

t/x2

p

Age

43.5 (11.5)

43.1 (11.0)

0.190

0.50

Gender (female)

31 (17)

23 (19)

0.900

0.34

Years in education

14.2 (2.3)

13.7 (2.3)

0.090

0.77

Diagnosis

1.792

0.77

Schizophrenia

29 (60.4%)

28 (66.7%)

Schizoaffective disorder

5 (10.4%)

5 (11.9%)

Bipolar disorder

5 (10.4%)

3 (7.1%)

Major depressive disorder

1 (2.1%)

2 (4.8%)

Other psychosis

8 (16.7%)

4 (9.5%)

Age at onset

26.4 (10.2)

25.9 (9.8)

0.002

0.96

Number of previous hospital admissions

3.9 (3.7)

5.0 (7.3)

0.915

0.34

Positive symptoms (SAPS total)

52.43 (29.13)

45.53 (22.89)

2.264

0.14

Negative symptoms (SANS total)

29.62 (17.57)

29.43 (11.42)

1.368

0.25

Global assessment of functioning score

67.44 (10.61)

67.29 (12.07)

0.564

0.46

Medication dosage (chlorpromazine equivalents)

395.10 (387.1)

599.35 (590.78)

-1.58

0.12


connectivity within the default network, affective network, and ventral attention network, as we previously identified significant differences in functional connectivity of these networks in patients with schizophrenia compared with healthy controls. In addition, we examined the cognitive control network due to the putative importance of this network in WM training. A more detailed MRI methodology is presented in the online Supplementary material.

Results

A total of 90 patients were recruited into the study, as illustrated in Fig. 1 (Consort diagram). Of 48 and 42 participants randomised to the intervention and control conditions respectively, no differences were observed between groups in either age, gender or years in education (see Table 1). Clinically, no differences were observed between groups at baseline in terms of diagnosis, age at onset, duration of illness, symptom severity or current antipsychotic medication dosage (measured in chlorpromazine equivalents).

Neuropsychological effects of CR

Mean scores for the intervention group and the control group for all cognitive variables analysed at baseline, 2 weeks post-treatment, and 3-6-month follow-up are presented in Table 2. At baseline (pre-treatment), no differences between the intervention group and the control group were observed for any cognitive measure (all p > 0.05).

After co-varying for baseline, significant differences in 2-week post-treatment follow-up scores were observed between the groups for both verbal and spatial WM task performance, and for matrix reasoning/performance IQ. A trend level difference was also observed for full scale IQ. On each of these measures, the intervention group significantly outperformed the control group. No other differences in cognitive performance were observed.

Mean differences between the intervention and control group at 2 weeks post-treatment follow-up remained significant at the 3-6-month follow-up, with two differences (see Fig. 2). Firstly, for spatial WM, although the mean differences between groups mirrors that seen at 2 weeks post-treatment follow-up (the intervention group showing near normal spatial WM performance of -0.05), this difference is no longer statistically significant. Secondly, significant differences were observed in verbal episodic memory between groups, such that the intervention group significantly outperformed the control group at this timepoint.

Effects of CR on social and occupational function, self-esteem and quality of life

No differences in self-esteem (Rosenberg Self Esteem Inventory) were observed at either 2 weeks postintervention or 3-6 months post-intervention follow-up assessment timepoints (see Table 3). Similarly, no differences in quality of life, (WHOQOL), were observed at either timepoint. In contrast, differences in social and occupational function were observed at both timepoints. At 2 weeks post-treatment follow-up, the intervention group had significantly better total scores on the UPSA-B than the control group. In a post hoc analysis these

Downloaded from https://www.cambridge.org/core. IP address: 80.82.77.83, on 21 Sep 2017 at 18:18:28, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S0033291717001982


Table 2. Comparison of cognitive performance and social function between the intervention and control groups at baseline, 2 weeks post-treatment, and 3-6 months post-treatment

General cognitive function

Baseline

2 week follow-up

3-6 month follow-up

CRT group

Mean (s.d.)

Control group

Mean (S.D.)

F

p

CRT group

Mean (S.D.)

Control group

Mean (S.D.)

F

p

CRT group

Mean (S.D.)

Control group

Mean (S.D.)

F

p

Similarities

8.56 (2.78)

8.48 (3.14)

0.02

0.89

10.01 (2.89)

8.80 (3.67)

1.66

0.2

9.87 (2.56)

8.47 (3.22)

1.3

0.26

Matrix reasoning

9.06 (3.04)

8.55 (3.07)

0.64

0.43

10.40 (2.48)

8.53 (3.30)

4.94

0.03

9.87 (2.61)

7.94 (2.08)

6.19

0.02

FIQ

91.92 (17.01)

90.28 (19.34)

0.18

0.67

102.08 (17.60)

91.18 (23.99)

3.12

0.08

99.07 (17.19)

87.88 (13.52)

3.35

0.08

Memory function

Logical memory immediate

7.69 (3.59)

7.00 (3.19)

0.91

0.34

9.12 (3.46)

8.27 (3.41)

0.95

0.34

10.00 (2.73)

6.88 (3.30)

4.22

0.05

Logical memory delayed

7.79 (4.15)

7.05 (3.47)

0.82

0.37

9.24 (3.80)

9.00 (3.30)

0.04

0.85

8.93 (3.79)

8.24 (3.56)

0.01

0.98

Letter-number sequencing

7.88 (3.55)

7.55 (3.92)

0.17

0.68

9.48 (3.65)

8.03 (2.93)

5.28

0.03

10.00 (3.64)

7.41 (2.94)

5.17

0.03

Spatial working memory (errors)

-0.56 (1.18)

-0.94 (1.17)

1.76

0.19

-0.53 (0.93)

-1.33 (0.91)

4.82

0.03

-0.06 (0.69)

-1.26 (1.33)

0.26

0.62

Executive function

WCST total errors

84.47 (16.52)

79.77 (16.12)

1.34

0.25

89.28 (17.59)

78.50 (18.10)

0.91

0.35

93.18 (13.27)

76.29 (18.86)

0.67

0.55

WCST perseveration errors

86.29 (10.52)

84.90 (16.24)

0.17

0.68

87.44 (12.70)

83.36 (15.07)

0.76

0.39

91.63 (8.80)

80.79 (12.78)

1.75

0.2

STROOP (colour word T score)

40.02 (10.09)

39.89 (10.60)

0.03

0.96

45.00 (11.44)

43.26 (8.05)

1.42

0.24

42.50 (10.32)

43.33 (10.02)

0.39

0.54

Trails A

48.56 (18.31)

49.20 (21.58)

0.02

0.88

39.62 (14.80)

41.20 (13.48)

0.07

0.8

37.13 (16.47)

41.56 (12.74)

0.28

0.6

Trails B

110.21 (65.70)

123.71 (70.82)

0.86

0.36

85.23 (46.00)

115.60 (74.46)

1.08

0.3

93.70 (53.88)

152.07 (101.05)

1.84

0.19

Social cognition

Reading the mind in the eyes

22.89 (5.23)

21.90 (5.57)

0.72

0.4

25.68 (3.84)

24.46 (4.62)

0.82

0.37

25.67 (4.69)

23.53 (5.13)

0.16

0.7


SOFAS, Social and occupational functional assessment scale; UPSA-B, UCSD performance-based skills assessment brief; ILS-PS, Independent living scale, problem-solving scale. For the 3-6 months follow-up period, the average length of time since training was 5 months.

Values in bold indicate significance p < 0.05.


Effectiveness of a low support, remotely accessible, CR training programme for chronic psychosis 7



Table 3. Differences between CR and Control Group in social and occupational function at baseline, 2 weeks post-treatment, and 3-6-month post-treatment

SOFAS (total)

CR Mean (s.d.)

Control Mean (S.D.)

F

p

Baseline

57.92 (11.42)

58.07 (13.2)

0.02

0.89

2-week post-treatment

68.96 (10.57)

62.4 (12.51)

3.71

0.06

3-6-month post-treatment

72.25 (11.44)

56.75 (10.72)

10.56

0.004

UPSA-B total scores

Baseline

66.18 (17.96)

66.87 (29.24)

0.06

0.94

2-week post-treatment

73.85 (10.31)

64.59 (17.78)

8.19

0.006

3-6-month post-treatment

73.92 (11.85)

70.67 (14.56)

0.65

0.43

ILS-PS

Baseline

37.02 (13.8)

37.0 (13.9)

0.00

0.99

2-week post-treatment

43.64 (7.37)

36.51 (14.74)

4.79

0.03

3-6-month post-treatment

43.43 (10.52)

35.79 (10.65)

4.35

0.05

Data: Mean (S.D.). SOFAS, Social and occupational functional assessment scale; UPSA-B, UCSD performance based skills assessment brief; ILS-PS, Independent living scale, problem-solving scale.

Values in bold indicate significance p < 0.05.

differences appear to have been driven by the financial subtest (F = 12.56; p = 0.001) as the communication scores did not show a significant change (p > 0.05). While significant differences on the financial subtest remained at the 3-6 months follow-up period (F = 4.48; p = 0.045), total score differences were no longer significant. Similarly, patients in the CR group showed trend level improvements in SOFAS scores at 2-week post-treatment follow-up, benefits which were statistically significant at 3-6-month follow-up.


Finally, the CR group at 3-6 months significantly outperformed the control group on our ILS problem-solving subtest at both follow-up time periods.

MRI results

N-back participant demographics

No significant differences were observed between the CR and control groups for age, gender, IQ, n-back WM accuracy or reaction time (n-back 2-back condition) at baseline (all p >0.05, see online Supplementary Table S2). At 2 weeks post-treatment follow-up, despite being underpowered, differences between groups in n-back performance were observed, such that CR patients had significantly slower reaction times and showed trend level (p = 0.052) improvements in accuracy during the task (see online Supplementary Table S3). There were no significant time x group interactions observed on number of ART outliers estimated for the n-back task across time 0 and time 1, across CR and control patient groups (p >0.05, see online Supplementary Table 2).

Resting-state participant demographics

Due to the potentially confounding effects of motion on the estimation of functional connectivity, mean scan to scan translation and rotation were calculated in MATLAB for each participant for each timepoint for the resting-state data. No significant time x group interactions were observed on mean translation, mean rotation, or number of ART outliers estimated for the resting-state fMRI data across time 0 and time 1, across CR and control patient groups (p > 0.05, see online Supplementary Table S4).

Effects ofCR on BOLD response during N-back task

No significant group x time interactions were observed on BOLD response during the n-back task (contrast 2-back > 0-back).

Effects of CR on functional connectivity during rest

We examined interactions between group (CR or control) and time (pre- or post-intervention) on connectivity of our six seed regions of interest (see online Supplementary Table S1). Significance was initially set at p < 0.05, FWE-corrected. Overall, we observed two group x time effects.

A group x time effect was observed on connectivity between the right precuneus and the left inferior parietal lobule (tmax = 6.53, n = 27; see online Supplementary Table S5 and Fig. 3). A second group x time effect was observed on connectivity between the left anterior cingulate and right midcingulate (tmax = 4.62, n = 27). Given that we examined functional connectivity across four resting-state networks, we also examined group x time effects at a more conservative threshold of p < 0.0125, FWE-corrected (i.e. correcting for the four networks analysed). The precuneus finding survived this additional correction, but the anterior cingulate finding did not (group x time effects on precuneus connectivity are reported at the p < 0.0125 threshold in).

Discussion

The study reports on the neuropsychological, functional, and neural effects of a low support CR programme for psychosis. While CR has received significant support in terms of its efficacy, the aim of the study was to ascertain whether patients could benefit from a programme involving only 1 h per week face-to-face contact with a clinician - the amount of contact often made available in psychological treatments. Using a RCT methodology involving stable community-based outpatients with psychosis, and a training programme that specifically targeted WM, three main outcomes were observed. Firstly, patients completing the CR training outperformed control group patients on measures of WM, episodic memory, and general cognitive ability, at 2 weeks post-treatment assessment and 3-6 months post-treatment assessment. Secondly, CR patients outperformed control patients on both skills-measured and rater-measured social and occupational function, again at both 2 weeks post-treatment follow-up and 3-6 months follow-up. Finally, in terms of changes in cortical activity following treatment, while no group x time effects were observed on BOLD response during an n-back task, strengthened connectivity was observed in two of four networks analysed (the default mode network and the affective network) in the CR group relative to the control group based on a resting state analysis. Collectively these data provide evidence at behavioural, occupational and cortical levels for the efficacy of even low-support 'e-health'-based CR interventions.

Low-support CR interventions

Following a meta-analysis of CR interventions for schizophrenia (Wykes et al. 2011), evidence of the benefits of CR is widespread, and also robust - with on average 0.45 s.d. improvements reported for patients receiving the intervention. The effects observed would appear to be relatively independent of the type of treatment provided - whether paper and pencil based, individual v. groups and (with few exceptions) the cognitive functions targeted. An important question, given this evidence of CR's efficacy, is how best to make CR available to patients in standard clinical care given the relatively limited resources available for psychological treatments in general, and in psychosis specifically. One approach to making psychological treatments more widely available has been via internet-based platforms - an approach described as 'eHealth', although the evidence base for these has been criticised as limited (Anthes, 2016). For patients with psychosis, the potential for adaptive computerbased training - training programmes that dynamically change in difficulty level according to patient ability - used in tandem with weekly therapist support rather than programmes that are purely 1:1, has received little attention to date. Given the cognitive and motivational challenges associated with psychosis, the feasibility of this approach is likely to be questioned by many clinicians. Here, however, in an unselected patient sample of chronic, middle-aged out-patients, more than half of the patients who began treatment were able to complete and in doing so, achieved a level of improvement comparable with that reported in the literature. The importance of these effects are likely to extend beyond WM-based training programmes to training of other cognitive deficits; the degree to which these, or indeed programmes targeting cognitive biases and metacognition, result in generalised improvement performance across multiple cognitive and social domains will be an important future research topic.

Neuropsychological effects

The specific targeting of WM in our programme was based on two factors - first, a desire to focus on an aspect of cognitive dysfunction whose underlying neural basis is relatively well established in schizophrenia (Lett et al. 2014; Constantinidis & Klingberg, 2016) and, secondly, the targeting of which may lead to more general benefits in cognition following even relatively little training. In a review of WM-based cognitive training in groups other than patients with psychosis, behavioural and neural studies have repeatedly associated WM ability with general cognitive ability, and WM training with general cognitive improvements, most likely by expanding the amount of information that can be represented and processed at the same time (Constantinidis & Klingberg, 2016). For schizophrenia, the decline in general cognitive ability reported in many patients makes targeting these deficits important. In the present study we observed that, in addition to improvements in untrained WM performance in the intervention group, improvements were also observed in episodic memory and in full scale IQ scores, the latter of which appeared to be driven by performance IQ (as measured by the WASI Matrix Reasoning subscale). That WM effects would generalise in terms of benefits to performance IQ rather than verbal IQ was not hypothesised, and may either reflect a null effect on verbal IQ or a difference in task sensitivity to change. However, a review of how WM training generalised to reasoning and intelligence tasks (Klingberg, 2010) indicated that WM effects were almost always reported for non-verbal rather than verbal reasoning tasks. In schizophrenia, only one WM-based CR study to date reported level changes in 'global cognition', but only at trend level and did not parse this finding into verbal and performance IQ (Hargreaves et al. 2015). In the present study, our findings appear more consistent with those reported in non-psychotic samples.

Effects on social and occupational functioning

As noted in a recent review by Green (2016), cognitive impairments have a significant impact on patientsfunctional status, hence the need to target these impairments. An important corollary of this view is that improved cognitive performance should be associated with improved functional outcomes. In the present study, we measured change in function both on the basis of observer rated estimates of function (SOFAS), and patient task performance (ILS and UPSA-B). Despite almost identical scores at baseline, the CR group showed significant improvements in social and occupational function at both 2 weeks post-treatment and 3-6-month post-treatment outcomes, depending on the measure used. We interpret these changes as providing reliable evidence of improvements given that the changes observed were in the same direction across all measures assessed. Given that improved cognitive performance has been reported in schizophrenia without leading to changes in functional outcome (Fiszdon et al. 2004), it is interesting to note that Rispaud et al. recently found that it was WM related improvements that best predicted functional improvements in psychosis (Rispaud et al. 2016).

These changes in function were observed in the absence of changes on the WHOQOL. Although we expected that improved cognition would be associated with benefits to quality of life, as for example Fizdon et al. recently reported (2016) (Fiszdon et al. 2004), no statistically significant changes in quality of life or selfesteem were observed. One interpretation of this is that these more psychological mediated benefits may take longer to become apparent. A high-intensity CR training study (Fisher et al. 2010; Subramaniam et al. 2014) targeting aspects of cognition including WM, observed significant gains in quality of life at 6-month follow-up but not before. Whether this longer follow-up period, or indeed the intensity or length of training were important factors leading to these benefits warrants further study.

Cortical effects of CR interventions

In recent reviews of cortical effects of CR training in schizophrenia (Ramsay & MacDonald, 2015; Isaac & Januel, 2016), a majority showed training related reductions in cortical activity, with increased activations reported in a number of regions, including lateral and medial pre-frontal cortices. We observed no significant group x time effects on BOLD response during an n-back task, despite using a larger sample size than all but two previous studies. One reason for this may be the range in amount of time in training completed by patients in our CR group, which we argue is likely to approximate 'real-world' clinical settings more accurately.

Few studies to date have investigated the effects of CR on functional connectivity in psychosis. In the Penades et al. (2013) study, a functional connectivity analysis based on the default mode network and the central executive network suggested reduced connectivity following training in the CR group. To our knowledge, ours is the first study to report on functional connectivity during resting state in psychosis. Based on an analysis of functional connectivity changes in three networks that we have previously studied in schizophrenia (Mothersill et al. 2016), we found increased activation in two of these networks - namely the default mode network and the affective network. This finding is consistent with findings in two separate studies of enhanced functional connectivity in healthy controls following WM training (Jolles et al. 2013; Takeuchi et al. 2013). The direction of results differs from the Penades et al. (2013) study, however, although as already noted, their analysis was task rather than resting-state based.

A recent review of the cortical effects of WM suggested that increased functional connectivity between frontal and parietal cortices, and strengthened inter and intra-network connectivity, represent two of the five biological factors underlying training induced increases in capacity [others including increased neural firing rate and changes in striatal dopamine release and signalling (Constantinidis & Klingberg, 2016)]. For example, Jolles et al. (2013) used fMRI to show that 6 weeks of WM training was associated with increased functional connectivity between frontal and parietal regions in healthy young adults (n = 15), and these changes were associated with increased accuracy on a WM task. Similarly, Thompson et al. (2016) used fMRI to show that 20 days of WM training was associated with increases in frontoparietal functional connectivity in healthy young adults (n = 20), and that larger increases in connectivity were related to larger improvements in WM performance. These studies suggest that post-training increases in frontoparietal functional connectivity are beneficial and associated with increased WM capacity.

Our data support the theory that WM capacity depends on, and is strengthened by, both inter- and intra-network connectivity. For the default mode network, we observed strengthened inter-hemispheric connectivity between the right precuneous and the left inferior parietal lobule. In addition, we also observed strengthened connectivity between the left anterior and right middle cingulate. In terms of Constantinidis & Klingberg's review (2016), rather than showing modulatory effects of pre-frontal cortex on parietal cortex, these findings suggest more intra-regional rather than inter-regional (e.g. topdown) strengthening of functional connectivity as the basis for increased capacity in patients with psychosis.

Study limitations

As already noted, when compared with cognitive training delivered either 1:1 or in groups, the dropout rate for this study was large, particularly for the intervention group. Similar dropout rates have recently been reported for other computer-based training initiatives (Fisher et al. 2009). While we have argued that, in a chronic patient sample, the fact that half of all patients were able to complete the training is relatively good, further studies of remotely accessed (i.e. non-clinic-based) training programmes will benefit from consideration of the factors that could improve acceptability and adherence. For example, efforts to address issues of motivation and adherence may benefit from the gamification' of training platforms (i.e. using gaming environments to improve interest and appeal; see Savulich et al. 2016; for examples). Of note, a majority of patients who dropped out of training did so in the first 1-2 weeks of training; on this basis either providing greater support at the start, or alternatively, more accurately identifying patients likely to experience difficulties may also offer a means of improving adherence.

Secondly, the amount of training undertaken by each patient varied widely, with many patients receiving significantly less than the 40 h of training often suggested as required to ensure durable benefits. For illustration, in addition to attending the weekly therapist support sessions, of the total sample recruited to the intervention arm of the study, the top 25% adherent group completed 32 or more hours or training, 50% of the sample completed 25 or more hours of training, and 75% of the sample complete 17 or more hours of training. Given the significant benefits in cognitive performance and function observed, this suggests that the benefits of cognitive training (at least for that focused on WM) requires less training to produce benefits than is typically advised. Of note, in a post hoc correlational analysis, no linear correlations between total time in training and cognitive benefits were observed on any outcome measure that showed significant change posttreatment. On the other hand, and offsetting these limitations in our view, is the low cost and potential increased accessibility of this training programme as part of routine clinical care. This is important because for those patients who can participate, the therapeutic benefits at a cognitive, social and cortical level, are comparable with other interventions with higher therapist staff costs (Wykes et al. 2011). However, the extent to which the increased training variability associated with remote training may affect generalisability of cognitive improvements remains to be investigated.

Finally, the number of patients in each arm of the study who underwent MRI was relatively large for CR studies, but nonetheless was likely to have been underpowered to identify subtle training related differences. While recent meta-analyses in this area highlight the cortical benefits of this training, the need for better powered imaging analysis remains. So also does the need to specify the relationship between the improved cognitive and social function observed and the changes in cortical function observed. Based on the present study, a reasonable hypothesis to pursue in future research is that increased cortical activity may be causally related to both improved cognitive performance and improved social function. Whether this is the case either directly or via mediation (e.g. whereby cognitive improvements mediate the effects of cortical changes on social functioning) remains to be investigated in a follow-on study.

Conclusion

CR training is reliably associated with improved cognitive performance on non-trained tasks, and with changes in cortical networks, particularly in terms of enhanced inter-hemispheric connectivity within prefrontal and parietal cortex. These benefits are, furthermore, associated with benefits to social and occupational functioning, aspects of psychosis related disability not addressed by pharmacological treatments. Given the limited resources available within many health services for delivering CR training, the evidence that even limited amounts of training, delivered remotely and with low support, can lead to such benefits is important. In the context of current knowledge about training, it provides further evidence that suggests CR can be made available in a manner that is relatively low cost and produce benefits that are durable.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0033291717001982.

Acknowledgements

We thank all patients and staff who participated in the collection of patient data. Recruitment of the patient sample was supported by a Health Research Board (Ireland) grant to GD.

Declaration of Interest

All authors confirm that they have no conflicts of interest in relation to this manuscript.

References

Andreasen NC (1984). Scale for the Assessment of Positive Symptoms (SAPS). University of Iowa: Iowa City.

Andreasen NC (1989). Scale for the Assessment of Negative Symptoms (SANS). University of Iowa: Iowa City.

Anthes E (2016). Mental health: There's an app for that. Nature 532, 20-23.

Baddeley A (2000). The episodic buffer: a new component of working memory? Trends in Cognitive Sciences 4, 417-423.

Baron-Cohen S, Jolliffe T, Mortimore C, Robertson M

(1997). Another advanced test of theory of mind: Evidence from very high functioning adults with autism or Asperger syndrome. Journal of Child Psychology and Psychiatry 38, 813-822.

Constantinidis C, Klingberg T (2016). The neuroscience of working memory capacity and training. Nature Reviews Neuroscience 17, 438-449.

Fett AKJ, Viechtbauer W, Penn DL, van Os J, Krabbendam L (2011). The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neuroscience and Biobehavioral Reviews 35, 573-588.

Fisher M, Holland C, Merzenich MM, Vinogradov S (2009). Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. American Journal of Psychiatry 166, 805-811.

Fisher M, Holland C, Subramaniam K, Vinogradov S (2010). Neuroplasticity-based cognitive training in schizophrenia: an interim report on the effects 6 months later. Schizophrenia Bulletin 36, 869-879.

Fiszdon JM, Bryson GJ, Wexler BE, Bell MD (2004). Durability of cognitive remediation training in schizophrenia: performance on two memory tasks at 6-month and 12-month follow-up. Psychiatry Research 125, 1 -7.

Glascher J, Gitelman D (2008). Contrast weights in flexible factorial design with multiple groups of subjects. SPM@ JISCMAIL. AC. UK.(https://www.jiscmail.ac.uk/cgibin/ webadmin?A2=ind0803&L=SPM&P=R16629)

Green MF (1996). What are the functional consequences of neurocognitive deficits in schizophrenia? American Journal of Psychiatry 153, 321.

Green MF (2016). Impact of cognitive and social cognitive impairment on functional outcomes in patients with schizophrenia. Journal of Clinical Psychiatry 77 (Suppl. 2), 8-11.

Green MF, Kern RS, Braff DL, Mintz J (2000).

Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the 'right stuff'Schizophrenia Bulletin 26, 119.

Hargreaves A, Dillon R, Anderson-Schmidt H, Corvin A, Fitzmaurice B, Castorina M, Robertson IH, Donohoe G (2015). Computerised working-memory focused cognitive remediation therapy for psychosis - a preliminary study. Schizophrenia Research 169, 135-140.

Haut KM, Lim KO, MacDonald A (2010). Prefrontal cortical changes following cognitive training in patients with chronic schizophrenia: effects of practice, generalization, and specificity. Neuropsychopharmacology 35, 1850-1859.

Heaton RK, Staff PAR (2003). Wisconsin Card Sorting Test: Computer Version 4-Research Edition (WCST: CV4)Psychological Assessment Resources: Lutz, FL.

Hubacher M, Weiland M, Calabrese P, Stoppe G, Stocklin M, Fischer-Barnicol D, Opwis K, Penner IK (2013).

Working memory training in patients with chronic schizophrenia: a pilot study. Psychiatry Journal, Article ID 154867, 8. doi: 10.1155/2013/154867.

Isaac C, Januel D (2016). Neural correlates of cognitive improvements following cognitive remediation in schizophrenia: a systematic review of randomized trials. Socioaffective Neuroscience & Psychology 6, 30054.

Jaeggi SM, Buschkuehl M, Jonides J, Perrig WJ (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences of the United States of America 105, 6829-6833.

Jaeggi SM, Buschkuehl M, Perrig WJ, Meier B (2010). The concurrent validity of the N-back task as a working memory measure. Memory 18, 394-412.

Jolles DD, van Buchem MA, Crone EA, Rombouts SA (2013). Functional brain connectivity at rest changes after working memory training. Human Brain Mapping 34, 396406.

Klingberg T (2010). Training and plasticity of working memory. Trends in Cognitive Sciences 14, 317-324.

Kundu B, Sutterer DW, Emrich SM, Postle BR (2013).

Strengthened effective connectivity underlies transfer of working memory training to tests of short-term memory and attention. Journal of Neuroscience 33, 8705-8715.

Lett TA, Voineskos AN, Kennedy JL, Levine B, Daskalakis ZJ (2014). Treating working memory deficits in schizophrenia: a review of the neurobiology. Biological Psychiatry 75, 361-370.

Li X, Xiao YH, Zhao Q, Leung AW, Cheung EF, Chan RC (2015). The neuroplastic effect of working memory training in healthy volunteers and patients with schizophrenia: implications for cognitive rehabilitation. Neuropsychologia 75, 149-162.

Lilienthal L, Tamez E, Shelton JT, Myerson J, Hale S (2013). Dual n-back training increases the capacity of the focus of attention. Psychonomic Bulletin and Review 20, 135-141.

Mausbach BT, Tiznado D, Cardenas V, Jeste DV, Patterson TL (2016). Validation of the UCSD Performance-based Skills Assessment (UPSA) in Hispanics with and without schizophrenia. Psychiatry Research 244, 388-393.

McAvinue LP, Golemme M, Castorina M, Tatti E, Pigni FM, Salomone S, Brennan S, Robertson IH (2013). An evaluation of a working memory training scheme in older adults. Frontiers in Aging Neuroscience 5, 20.

McCarthy H, Skokauskas N, Mulligan A, Donohoe G, Mullins D, Kelly J, Johnson K, Fagan A, Gill M, Meaney J, Frodl T (2013). Attention network hypoconnectivity with default and affective network hyperconnectivity in adults diagnosed with attention-deficit/hyperactivity disorder in childhood. JAMA Psychiatry 70, 1329-1337.

Mothersill O, Tangney N, Morris DW, McCarthy H, Frodl T, Gill M, Corvin A, Donohoe G (2016). Further evidence of alerted default network connectivity and association with theory of mind ability in schizophrenia. Schizophrenia Research 184, 52-58.

Patel A, Knapp M, Romeo R, Reeder C, Matthiasson P, Everitt B, Wykes T (2010). Cognitive remediation therapy in schizophrenia: cost-effectiveness analysis. Schizophrenia Research 120, 217-224.

Penades R, Pujol N, Catalan R, Massana G, Rametti G, Garcia-Rizo C, Bargallo N, Gasto C, Bernardo M, Junque C (2013). Brain effects of cognitive remediation therapy in schizophrenia: a structural and functional neuroimaging study. Biological Psychiatry 73, 1015-1023.

Ramsay IS, MacDonald AW (2015). Brain correlates of cognitive remediation in schizophrenia: activation likelihood analysis shows preliminary evidence of neural target engagement. Schizophrenia Bulletin 41, 1276-1284.

Reitan RM, Wolfson D (1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation, vol. 4. Reitan Neuropsychology: Tucson, Arizona.

Rispaud SG, Rose J, Kurtz MM (2016). The relationship between change in cognition and change in functional ability in schizophrenia during cognitive and psychosocial rehabilitation. Psychiatry Research 244, 145-150.

Robbins TW, James M, Owen AM, Sahakian BJ, McInnes L, Rabbitt P (1994). Cambridge neuropsychological test automated battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers. Dementia and Geriatric Cognitive Disorders 5, 266-281.

Rosenberg M (1965). Rosenberg self-esteem scale (RSE). Acceptance and commitment therapy. Measures package, 61.

Rudebeck SR, Bor D, Ormond A, O'Reilly JX, Lee AC (2012). A potential spatial working memory training task to improve both episodic memory and fluid intelligence. PLoS ONE 7, e50431.

Rybarczyk B (2011). Social and occupational functioning assessment scale (SOFAS). In Encyclopedia of Clinical Neuropsychology (ed. J. Kreutzer, J. DeLuca, and B. Caplan), pp. 2313-2313. Springer: New York.

Salminen T, Strobach T, Schubert T (2012). On the impacts of working memory training on executive functioning. Training-Induced Cognitive and Neural Plasticity 6, 166.

Savulich G, Piercy T, Fox C, Suckling J, Rowe JB, O'Brien JT, Sahakian B (2016). Cognitive Training Using a Novel Memory Game on an iPad in Patients with Amnestic Mild Cognitive Impairment (aMCI).

Stevens AA, Tappon SC, Garg A, Fair DA (2012). Functional brain network modularity captures inter-and intraindividual variation in working memory capacity. PLoS ONE 7, e30468.

Stroop JR (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology 18, 643-662.

Stroop JR (1992). Studies of interference in serial verbal reactions. Journal of Experimental Psychology: General 121, 15.

Subramaniam K, Luks TL, Garrett C, Chung C, Fisher M, Nagarajan S, Vinogradov S (2014). Intensive cognitive training in schizophrenia enhances working memory and associated prefrontal cortical efficiency in a manner that drives long-term functional gains. Neuroimage 99, 281-292.

Takeuchi H, Taki Y, Nouchi R, Hashizume H, Sekiguchi A, Kotozaki Y, Nakagawa S, Miyauchi CM, Sassa Y, Kawashima R (2013). Effects of working memory training on functional connectivity and cerebral blood flow during rest. Cortex 49, 2106-2125.

Thompson TW, Waskom ML, Gabrieli JDE (2016). Intensive working memory training produces functional changes in large-scale frontoparietal networks. Journal of Cognitive Neuroscience 28, 575-588.

Wechsler D (1998). Wechsler Memory Scale, 3rd edn (WMS-III). The Psychological Corporation: New York.

Wechsler D (1999). Wechsler Abbreviated Scale of Intelligence. The Psychological Corporation: Harcourt Brace & Company: New York, NY.

Wexler BE, Anderson M, Fulbright RK, Gore JC (2000).

Preliminary evidence of improved verbal working memory performance and normalization of task-related frontal lobe activation in schizophrenia following cognitive exercises. American Journal of Psychiatry 157, 1694-1697.

World Health Organization (1996). WHOQOL-BREF: introduction, administration, scoring and generic version of the assessment: field trial version, December 1996.

Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P (2011). A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. American Journal of Psychiatry 168, 472-485.

Wykes T, Reeder C (2005). Cognitive Remediation Therapy for Schizophrenia. Brunner Routledge: London.

Schizophrenia Bulletin vol. 41 no. 1 pp. 250-258, 2015 doi:10.1093/schbul/sbt232

Advance Access publication January 20, 2014

Neuroplasticity-Based Auditory Training Via Laptop Computer Improves Cognition in Young Individuals With Recent Onset Schizophrenia

A/TfdiccQ TTigIiai*1,2          T apwv1             C1 oiT/^i*3 A dil/xv T aa1 T noni/xl Uarrlorirl3 Tara NTi^nrloin3

essaset ,aceoewy ,d^neivnaiiei ,seyee ,j.  aiiie  agiaiid , ara  eiiddm ,

Danielle Schlosser1, Lien Pham3, Tara Miskovich3, and Sophia Vinogradov*,1,2

'Department, of Psychiatry, University of California, San Francisco, CA; 2Department of Psychiatry, San Francisco Department of Veterans Affairs Medical Center, San Francisco, CA; 3Department of Psychiatry, University of California, Davis, CA

*To whom correspondence should be addressed; Department of Psychiatry, 116A—San Francisco Department of Veterans Affairs Medical Center, 4150 Clement Street, San Francisco, CA 94121, US; tel: 415-221-4810 ext. 3106, fax: 415-379-5574, e-mail: Sophia.vinogradov@ucsf.edu

Objective: Cognitive deficits that characterize schizophrenia are present in the prodrome, worsen with illness onset, and predict functional outcome. Cognitive dysfunction is thus a critical target for early intervention in young individuals with recent onset schizophrenia. Method: This

Key words: cognitive remediation/cognitive training/motivation/first-episode schizophrenia/early psychosis

Introduction

Early intervention in schizophrenia is a critical treatment goal, and cognitive dysfunction is arguably a key treatment target.1,2 Cognitive deficits are present during the prodrome and worsen with the onset of psychosis.3,4 Importantly, processing speed and verbal memory at first episode predict community functioning 7 years later.5

Two prior randomized controlled trials (RCTs) of cognitive remediation have been conducted in early schizophrenia—one using paper-and-pencil methods6 and one using a 2-year combination of computerized remediation plus social skills groups7—with respective effect sizes of 0.13 and 0.60 on global cognition.8 In both studies, active and control groups differed in the number of hours of clinician contact and/or received different adjunctive psychotherapies, and some assessments were conducted by staff not blind to group assignment, making it difficult to determine the active ingredients of treatment response.

To address these issues, we performed a 2-site double-blind RCT comparing approximately 40 hours of computerized auditory processing and verbal learning training to 40 hours of commercial computer games (CG) in adolescents and young adults who were within 5 years of psychosis onset (mean illness duration less than 2 years). The cognitive training was derived from basic research on neuroplasticity and shows demonstrable effects in frontotemporal networks in adults with persistent schizophrenia.9-12 A heavy schedule of auditory perceptual training is embedded within increasingly complex auditory/verbal working memory exercises in order to improve the temporally detailed resolution of auditory cortical representations and downstream working memory processes. In a prior double-blind RCT, adults with persistent schizophrenia (average age of 40 years), showed significant gains in verbal learning/memory and global cognition after 50 hours of training compared with commercial CG played for 50 hours. Gains were accompanied by adaptive neurobiological changes not seen in the control group.9-13 Four other studies have investigated this form of training in schizophrenia, but methodologies and results have been inconsistent across studies.14-17 While some of the findings have been promising, the discrepancies indicate the need for additional research.

© The Author 2014. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com


In this study, we investigated the effects of this form of auditory training (AT) in young individuals after illness onset, delivered at home on a portable computer rather than in a laboratory or clinical setting, in keeping with a stigma-free and recovery-oriented treatment model. Participants were loaned laptop computers with the necessary software and participated in the intervention on their own schedule, with participation verified by the software. We hypothesized that participants would show significant improvement in verbal learning/memory and general cognition, consistent with our findings in adults with persistent illness. We also examined two behavioral predictors of training response—baseline motivational system functioning (reward anticipation) and auditory system target engagement. Because anticipatory reward processing and motivation are known to enhance learning and memory performance,18-20 we hypothesized that self-ratings of higher reward anticipation at baseline would predict better response to training (larger cognitive gains). We additionally hypothesized that individuals who showed early evidence of improved auditory processing speed within the training exercises—ie, successful engagement with the auditory system training target— would show greater cognitive gains (see also Murthy et al16).

Method

Participants

Eighty-six subjects completed the study protocol in our university-based early psychosis clinics at University of California, San Francisco and University of California, Davis (ClinicalTrials.gov NCT00694889). Subjects were recruited via presentations/flyers and through clinician referrals, and they met the following inclusion/exclusion criteria: (1) dagnosis of schizophrenia, schizophreniform, or schizoaffective disorder; (2) onset of first psychotic episode within past 5 years; (2) good general physical health; (3) age 14-30 years; (4) fluent and proficient in English; (5) intelligence quotient (IQ) > 70; (6) no neurological disorder; and (7) no substance dependence in past year. All subjects had achieved outpatient status for at least 3 months and, among participants taking psychiatric medications (N = 81), were on a stable dose for at least one month prior to participation. Five participants did not take psychiatric medications.

Participants aged 18 and older gave written informed consent, while those younger than age 18 provided assent, with written parental/legal guardian consent. Baseline assessments were conducted prior to randomization. Subjects were stratified by IQ, gender, and symptom severity and randomly assigned to AT or to the CG control condition (CONSORT diagram in figure 1). Subjects were loaned laptop computers and participated in the intervention at home, except for 1 training subject who preferred to participate in the laboratory. Subjects were asked to participate for 40 hours (1 h/d, 5 d/wk, for 8 wk), followed by posttraining assessments.

Participants were contacted 1-2 times per week by telephone to discuss progress. Coaching was provided if a participant indicated difficulty in completing the recommended number of hours/week (eg, goal-setting; discussion of scheduling; setting an alarm, and using reminders). At a “check-in” appointment after every 10 sessions completed, the same coaching was provided and participants were paid $5 for each completed hour, $20 for every 10 sessions, and $30 after 40 hours, as well as $20 per assessment appointment. Mean training time was 34.65 hours (SD = 9.82) across both groups (table 1). While in the trial, participants received treatment by outside providers or clinic personnel not involved in the study (psychoeducation, psychotherapy, adjustments in medications as clinically indicated). Demographic characteristics and medications are presented in tables 1 and 2.

Cognitive Training Program and CG Control Condition The cognitive training program was provided by Posit Science Corporation and has been described previously.9 It consists of computerized exercises designed to improve speed and accuracy of auditory information processing while engaging auditory and verbal working memory. This training approach is based on evidence that schizophrenia is characterized by widespread disturbances in frontotemporal neural systems subserving auditory processing and verbal memory.22,23 Exercises continuously adjust difficulty level to maintain an 80%-85% correct performance rate in order to engage the user in a dense reward schedule and drive successful learning. Correct trials are rewarded with points and animations. In each session, a participant works with 4 of 6 exercises for 15 minutes per exercise. Compliance is monitored by electronic data upload.

The CG control condition allows for maintenance of a double-blind trial design and controls for effects of computer exposure, contact with research personnel, monetary payments, and nonspecific engagement of attention, executive functions, and motivation. Control subjects rotated


Fig. 1. Consort diagram of subjects with recent onset schizophrenia who received computerized auditory training (AT) and patients who played computer games (CG).

Table 1. Baseline Characteristics of Subjects With Recent Onset Schizophrenia Who Received Computerized Auditory Training (AT) and Subjects Who Played Computer Games (CG)

AT (N = 43)

CG (N = 43)

t

df

P

Mean

SD

Mean

SD

Male/femalea

31/12

33/10

Age (range 16-30)

21.70

3.26

20.74

3.37

-1.34

84

.19

Education

12.88

1.60

12.86

2.10

-0.06

84

.95

WASI IQb

102.63

12.12

100.67

15.06

-0.66

84

.51

PANSS totalc

57.95

12.72

59.60

14.33

0.55

80

.58

Strauss Carpenter

7.83

2.34

8.00

2.76

0.31

80

.76

Global functioning role

4.79

2.47

4.71

2.14

-0.16

80

.88

Global functioning social

5.72

1.30

5.74

1.43

0.07

80

.95

Hours of training

32.93

10.45

36.37

8.94

1.64

84

.10

Months of illness (range 1-60)

18.87

15.64

20.26

17.45

0.39

84

.70

aChi square (1, N = 86) = 0.24, P = .62.

bWechsler Abbreviated Scale of Intelligence (WASI). cPositive and Negative Syndrome Scale.

through a series of 16 different commercially available games (supplementary table 1) for the same number of hours as training subjects, playing 4-5 games on any given day.

Assessment Procedures

All assessment staff were blind to group assignment. Cognitive assessment staff were trained and monitored at each site on manualized assessment procedures by the same senior researcher (M.F.) to ensure cross-site consistency (the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery showed an intraclass correlation of 0.88 in a multisite RCT).24 Clinical assessment staff were trained and observed by expert clinical supervisors at each site (R.L.L., J.D.R., and T.N.). Subject eligibility was determined in regular reliability rounds. Interrater reliability was calculated from staff ratings of training tapes, with an average intraclass correlation across sites of 0.83 for symptom ratings and an average kappa value of 0.95 for diagnostic agreement.

Table 2. Medication Regimens of Study Participants

AT (N = 43)

CG (N = 43)

Total

Test Statistic

P Value

Antipsychotic medicationa

First generation (N)

1

1

2

A2(1) = 0.00

1.00

Second generation (N)

37

34

71

A2(1) = 0.73

.39

Multiple (N)

2

3

5

A2(1) = 0.21

.64

No antipsychotic (N)

3

5

8

A2(1) = 0.55

.46

Other psychiatric medication

Antidepressants or mood stabilizers (N)

8

12

20

A2(1) = 1.04

.31

Benzodiazepines (N)

5

9

14

A2(1) = 1.37

.24

Other medication measures

Chlorpromazine equivalentsb

235.47 (155.96)

256.36 (136.75)

t (75) = 0.62

.53

Changes in medication while in study

12

13

A2(1) = 0.06

.81

Note: AT, auditory training group; CG, computer games control group.

aFirst generation antipsychotic medication = halperidol, perphenazine, thiothixene, trifluoperazine;

Second generation antipsychotic medication = aripiprazole, clozapine, olanzapine, quetiapine, risperidone, ziprasidone. bMean and SD of chlorpromazine equivalents.21


Eligibility diagnoses were determined using the Structured Clinical Interview for DSM-IV Symptoms and functioning were assessed with the Positive and Negative Syndrome Scale25 (PANSS), Strauss Carpenter Outcome Scale (3 clinician-rated items for number of weekly social contacts, days hospitalized, and proportion of time engaged in work/school),26 and Global Functioning: Role and Social Scales (each containing a single clinician-rated item ranging from 1-10 developed for the late adolescent/young adult early psychosis population).27 An abbreviated battery of mAtRICS-recommended measures28 was administered (table 3). The Tower Test from the Delis-Kaplan Executive Function System (D-KEFS)29 was used in place of the Neuropsychological Assessment Battery (NAB) Mazes. Raw scores were converted to z scores using age-appropriate normative data provided in testing manuals and age-appropriate, published normative data for Trails A,30 Category Fluency,31 and the Brief Visuospatial Memory Test-Revised (BVMT-R).32 A 6-point Likert scale was used to assess level of enjoyment.

All primary outcome measures were distinct and independent from tasks practiced during training. Alternate forms of the Hopkins Verbal Learning Test-Revised (HVLT-R) and BVMT-R were administered and counterbalanced at baseline and posttraining.

Baseline Reward Anticipation

Reward anticipation was measured using the Temporal Experience of Pleasure Scale, which assesses anticipatory and consummatory pleasure (see Gard et al33 for psychometric properties). Eighteen items are rated on a scale of 1 (very false for me) to 6 (very true for me). An example of an anticipatory item is “When something exciting is coming up in my life, I really look forward to it.” Accumulated evidence indicates that these are distinct neurobehavioral processes34; anticipatory pleasure appears closely linked to motivation and goal-directed behavior. Because prior research indicates that schizophrenia patients have a deficit in anticipatory pleasure and because successful learning requires intact brain motivational systems,18-20,35 we hypothesized that individuals with higher reward anticipation at baseline would show greater cognitive gains after training.

Target Engagement: Gains in Auditory

Processing Speed

Subjects’ ability to demonstrate target engagement—ie, to show improvement in auditory cortical processing efficiency—was monitored at baseline and after 20 hours of training using auditory processing speed. This measure consists of a time-order judgment of a sequence of 2 frequency-modulated tones and is considered a measure of successive signal interference/forward and backward masking. Performance threshold was determined using a dual staircase method based on the ZEST algorithm36 that adaptively modifies the interstimulus interval (ISI) between tones and the tone duration, which is held equal to the ISI. The resulting score was the number of milliseconds of ISI (and tone duration) at which the subject correctly performed 66% of trials, allowing for a precise measure of psychophysical threshold under moderate perceptual challenge, as outlined in prior research on reliable threshold assessments in auditory processing.37

Planned Analyses

We performed data analysis on all subjects completing baseline and posttraining assessments (N = 86), regardless of hours of intervention (see figure 1), and an intent-to-treat analysis using last observation carried forward on all randomized subjects (N = 121). Based on

Table 3. Scores on Cognitive Domains, Symptoms and Functional Outcomes Before and After Intervention for Subjects With Recent

Onset Schizophrenia Who Received Computerized Auditory Training (AT) and Subjects Who Played Computer Games (CG)

Outcome Measuresa

AT (N = 43)

CG (N = 43)

F>

P

Effect Size

Baseline

Post

Baseline

Post

Mean

SD

Mean

SD

Mean

SD

Mean

SD

Global cognition

-0.86

0.73

-0.46

0.73

-0.92

0.89

-0.87

1.00

11.07

<0.01

0.73

Speed of processing

-0.66

0.76

-0.27

0.80

-0.63

0.79

-0.55

1.11

2.50

0.12

0.33

Working memory

-0.36

0.78

-0.12

1.01

-0.62

1.13

-0.41

1.00

0.01

0.92

0.04

Verbal learning

-1.45

1.41

-1.29

1.38

-1.55

1.33

-1.88

1.80

2.45

0.12

0.43

Verbal memory

-1.68

1.54

-1.10

1.32

-1.46

1.51

-1.79

1.78

9.35

<0.01

0.69

Visual learning

-1.16

1.62

-0.68

1.46

-1.09

1.51

-1.04

1.66

3.27

0.07

0.33

Visual memory

-1.06

1.78

-0.60

1.43

-1.31

1.86

-1.07

1.84

0.66

0.42

0.14

Problem solving

-0.36

0.76

0.29

0.86

-0.38

0.81

-0.12

1.00

4.83

0.03

0.46

PANSS totalc

57.95

12.72

53.56

12.48

59.60

14.33

55.86

14.18

0.45

0.51

0.06

Strauss Carpenter

7.83

2.34

8.13

2.27

8.00

2.76

8.55

2.65

0.28

0.60

-0.10

Global functioning role

4.79

2.47

4.79

2.63

4.71

2.14

4.71

2.43

0.01

0.91

0.00

Global functioning social

5.72

1.30

5.79

1.15

5.74

1.43

5.95

1.62

0.38

0.54

-0.11

aCognitive measures were transformed to z scores using normative data of healthy samples. Global cognition (average z score across all measures); speed of processing (Trail Making Test Part A; category fluency animal naming); working memory (letter-number span; WMS-III spatial span); verbal learning and verbal memory (HVLT-R immediate and delayed recall); visual learning and visual memory (BVMT-R immediate and delayed recall); problem solving (D-KEFS Tower Test). Symptoms (PANSS) and functional outcome measures (Strauss Carpenter; global functioning role and social) are clinician ratings.

bRepeated measures ANCOVA, condition-by-time interaction, controlling for site, age, and hours of training.

c Positive and Negative Syndrome Scale.


our previous finding that medication-induced anticholinergic burden adversely affects training response,38 we excluded from analysis 2 subjects (1 in each condition) who were prescribed an increased dose of benztropine mesylate while in the study. All variables were screened and normally distributed after winsorising of outlying values.

Chi square was used to test for group differences in attrition rate. Groups were compared on change in cognitive measures, symptoms, and functional outcome ratings using repeated measures ANCOVA, controlling for site, age, and hours of training. Effect sizes (Cohen’s d) were computed using the mean change scores of the AT and CG groups (posttraining minus baseline) and the change score SDs of each group. All measures are listed in table 3. Paired sample’s t-test tested for significant gains in auditory processing speed. Pearson’s correlations tested whether baseline reward anticipation and improvement in auditory processing speed were associated with gains in global cognition.

Results

Adherence to Cognitive Training via Laptop

Nineteen out of 63 (30%) AT subjects withdrew from the study compared with 14 of 58 (24%) CG subjects, a nonsignificant difference, X2 (1, N = 121) = 0.55, P = .46. Mean enjoyment ratings were 3.78 (SD = 0.90, range = 1.83-6) in At subjects, and 3.88 (SD = 1.05, range = 2-6) in CG subjects, a nonsignificant difference, t (70) = 0.45, P = .66; rating of 4 = “slightly enjoyed” on a scale of 1 (“extremely disliked”) to 6 (“extremely enjoyed”).

We compared study completers with those who withdrew and found no significant differences in demographic variables, baseline symptoms and functioning, or reward anticipation (supplementary table 2). All baseline cognitive differences were nonsignificant with the exception of working memory. Study completers showed lower baseline working memory (M = -0.49, SD = 0.98) relative to participants who withdrew (M = -0.10, SD = 0.60), (117) = 2.62, P = 0.01.

Cognitive Outcome Measures

There were no differences between groups in baseline symptom severity, functioning, cognitive measures, or medication regimens (table 2). A multivariate ANCOVA on cognitive change across all cognitive measures showed a significant effect of condition (F = 2.62, df = 1, 75, = 0.02). Repeated measures ANCOVA revealed significant condition-by-time interactions for global cognition, verbal memory, and problem solving and a difference at trend level in visual learning (table 3, figure 2). The results were the same with and without covarying for site, age, and hours of training.

An intent-to-treat analysis revealed the same significant condition-by-time interactions in global cognition (mean z score change AT = 0.28, SD = 0.40, CG = 0.04, SD = 0.45, F = 10.48, df = 1, 114, P < .01), verbal memory (mean z score change AT = 0.41, SD = 1.04, CG = -0.25,


SD = 1.25, F = 9.29, df = 1, 114, P < .01), problem solving (mean z score change AT = 0.45, SD = 0.80, CG = 0.19, SD = 0.73, F = 4.47, df = 1, 114, P < .04), and a difference at trend level in visual learning (mean z score change AT = 0.34, SD = 1.00, CG = 0.04, SD = 1.25, F = 3.04, df = 1, 114, P < .08).

Cognitive Effect Sizes of the Intervention

In study completers, strong positive effects for training were found in global cognition, verbal memory, and problem solving (table 3). Effect sizes were comparable with our findings in middle-aged adult subjects who were more cognitively impaired at baseline.9 In all randomized subjects, effect sizes were in the moderate range for global cognition (d = 0.56) and verbal memory (d = 0.57), and in the small range for problem solving (d = 0.34).

Post hoc analyses of verbal memory revealed that the increase in AT subjects was significant, t (42) = 3.19, P = .003, while the decline in the computer CG group approached trend level, t (42) = 1.51, P = .14. A similar nonsignificant decline in verbal learning and memory was shown in middle-aged CG subjects with persistent schizophrenia.9

Symptom and Functional Outcome Measures

Four subjects (3 AT, 1 CG) did not complete all clinical measures at both time points. There were no significant condition-by-time interactions on the PANSS total or subscales, Strauss Carpenter Scale, or Global Functioning Role and Social scales. There was a main effect of time in the PANSS Total and General Psychopathology Subscales. Both groups showed a small decrease in PANSS Total (Mean Rating Change = -4.05, SD = 11.47, F = 6.23, df = 1, 81, P = .02), PANSS General Psychopathology (Mean Rating Change = -2.83, SD = 5.88, F = 7.57, df = 1, 81, P < .01), and no significant change in PANSS Positive or Negative Symptoms. On measures of functional outcome, all main effects of time were nonsignificant. The results remained the same with and without imputation (last observation carried forward/backward) of the 4 subjects’ missing values.

The intent-to-treat analysis revealed the same findings: a main effect of time in the PANSS Total (Mean Rating Change = -2.79, SD = 9.69, F = 4.84, df = 1, 114, P = .03) and General Psychopathology Subscales (Mean Rating Change = -1.95, SD = 9.69, F = 5.04, df = 1, 114, P = .03), and no significant main effects of time or condition-by-time interactions in PANSS Positive or Negative Symptoms, or functional outcome measures.

Baseline Reward Anticipation and Cognitive Gains

The reward anticipation measure was available on 24 AT and 23 CG subjects. In the AT group, baseline reward anticipation was significantly associated with gains in global cognition (r = 0.52, P < .01) and verbal memory (= 0.51, P = .01) and remained significant controlling for hours of training (supplementary figure 1). These associations were not significant in the CG group. (-0.20 < < 0.09, 0.36 < P < .68).

Target Engagement and Cognitive Gains

The auditory processing speed measure was available at the 2 time points on 30 of 40 AT subjects. There was a highly significant decrease from baseline (Mean = 98.07ms, SD = 42.86) to 20 hours (Mean = 63.20 ms, SD = 37.45), t (29) = 5.07, P < .001), indicating that subjects became more efficient at rapid processing of successive auditory stimuli (target engagement). This improvement was significantly associated with gains in global cognition (r = -0.47, P < .01) (supplementary figure 2).

Discussion

Young individuals with recent onset schizophrenia who completed up to 40 hours of cognitive training via laptop computer at home showed significant increases in global cognition, verbal memory, and problem solving, compared with individuals who played up to 40 hours of CG. Participants in both groups—the majority of whom were enrolled in university-based early psychosis clinics— showed small improvements in symptoms. These findings suggest that the cognitive deficits of schizophrenia can be addressed early in the course of illness using cognitive training delivered via a portable computing device. This is particularly important as emerging data show that current clinical interventions in early psychosis may not differentially impact outcome 5 years later.39 Symptom reduction and psychosocial support alone are probably not sufficient as treatment targets if our ultimate goal is to disrupt the deteriorating course of schizophrenia. We posit that cognition is a third critical factor that must be addressed vigorously and systematically in young individuals.

Young recent onset subjects showed a similar profile of training-induced cognitive improvement as was observed in our prior study of persistently ill adults9; similarly, young “computer games” subjects showed a decrease in HVLT performance approaching trend level. To the best of our knowledge, this is the first report of decreases in verbal memory after commercial CG in a young clinical group. In a sobering study with healthy school-age children, Dworak et al40 found a significant reduction in verbal memory after a single day of exposure to voluntary excessive computer game playing and television. We hypothesize that the nonspecific visuospatial processing from an intensive schedule of CG resulted in competitive interference for limited neural resources, causing worse performance on the HVLT. These findings require replication, as they have important implications for young individuals with schizophrenia.

We also examined 2 predictors of treatment response: reward anticipation (motivational system functioning) and auditory target engagement. We found that baseline ratings of reward anticipation were significantly associated with cognitive gains in AT subjects, but not in the CG control group, even after controlling for number of hours of training. This finding is consistent with growing knowledge on the role of interindividual differences in reward processing in successful learning and plasticity mechanisms.18-20,35 Indeed, Tas et al41 recently reported that motivation and metacognition predicted learning potential in people with remitted schizophrenia.

Additionally, we found that participants with the largest improvement in auditory processing speed after 20 hours of exposure to training showed the greatest gains in global cognition at posttraining. Improvement in auditory processing speed indicates successful target engagement of the auditory system with an improvement in cortical processing efficiency. This is consistent with our prior study,9 and with reports from Murthy et al16 and Surti et al,42 demonstrating a relationship between processing speed gains and generalization of training effects. The National Institute of Mental Health recommends incorporating measures of target engagement into treatment trials in order to provide very early signs of potential failure or success of an intervention. Our current findings suggest that target engagement may be measurable mid-way through treatment (and perhaps even earlier). Early gains in auditory processing speed likely serve as an indirect measure of training-induced plasticity and increased efficiency in the neural systems subserving auditory encoding. Experiments in our persistent illness subjects indicate that training induces an increase in evoked responses in auditory cortex, accompanied by changes in activation patterns in prefrontal cortex that are associated with cognitive improvement.11

To date, few variables reliably predict response to cognitive training in schizophrenia: not type of training nor participant characteristics.8 While our results require replication, they do suggest that— at least for the highly targeted “neural system” cognitive training that we are studying—the inherent “plasticity potential” of the brain as assessed via intact reward systems and engagement of plasticity mechanisms in neural system targets of interest may be much more important predictors of treatment response than participant demographics. Indeed, these data begin to open the door to the possibility of truly personalized medicine.

There are several limitations to our study. Foremost is the lack of immediate impact on everyday functioning, which likely reflects the short study duration, or perhaps the lack of social cognition training or psychosocial interventions that enhance generalization.7,43,44 We are currently investigating whether cognitive gains persist 6 months beyond the intervention and are associated with functional improvement, as previously observed in persistently ill adults.45 Second, subjects were drawn from 2 university clinics, were of average IQ, and are not representative of community settings. We do not know if this intervention would be as effective in individuals with lower IQ or under 16 years of age. Subjects were provided monetary compensation for participation in the trial, which limits our knowledge about acceptability and adherence in real-world settings where payment will not be provided. We did not place restrictions on medication regimens during study participation, and, while there is no evidence of significant differences between the two groups, we cannot rule out medication effects on the response to the intervention. We also did not control for at-home exposure to video games or other computerized activities, which may represent a confounding variable. Further, the lack of significant improvement in MATRICS processing speed and working memory raises important questions for the design of future cognitive training programs, the selection of key training targets, and the as-yet-unknown association linking gains across cognitive domains. Finally, while the attrition rate is similar to other behavioral treatment studies in recent onset schizophrenia,46 it is still quite large. Future efforts must focus on the pragmatic effectiveness of this intervention and on enhancing appeal and increasing participant adherence.

In sum, results indicate that a neuroscience-informed approach to cognitive training in young individuals with schizophrenia, using a portable computer, can generate significant improvements in cognition, a finding with promising implications for the long-term course of illness. Given limited mental health resources and the rapid expansion of portable digital technology, further well-controlled trials of cognitive training via mobile devices is an important area for future investigation. If our data are replicated, a number of critical questions will need to be addressed. What are the necessary and sufficient elements of training that generate optimal and enduring functional gains in young individuals? How do we design the intervention to be optimally motivating and rewarding? How should it best be disseminated in real-world treatment settings? And most importantly, how can it be personalized and combined with psychosocial treatments so that every young person with schizophrenia can look forward to a productive, stable, and maximally fulfilling life course?

Supplementary Material

Supplementary material is available at http://schizophre-niabulletin.oxfordjournals.org.

Funding

The Stanley Medical Research Institute (06TAF-972); the Laszlo Tauber Foundation; the National Institute of Mental Health (5R01MH081051); the San Francisco Department of Veterans Affairs Medical Center.

Acknowledgments

The cognitive training software used in this study was supplied to the first author free of charge by Posit Science. Dr Vinogradov is a paid consultant to Brain Plasticity Inc. (now a division of Posit Science), a company with a commercial interest in the cognitive training software used in this study. None of the other authors have any financial interest in Brain Plasticity Inc. or Posit Science. Drs Fisher, Loewy, Carter, Ragland, Niendam, Schlosser, and Vinogradov and Ms Lee, Ms Pham, and Ms Miskovich report no conflicts of interest. Drs Loewy, Carter, Ragland, Niendam, Schlosser, and Vinogradov have received grants or research support from the National Institute of Mental Health. Dr Vinogradov serves on advisory boards for Genentech, Hoffman-LaRoche, and Envivo. Dr Carter has served on the advisory board of Merck, Lilly, Pfizer, Roche, and Servier and has received research funding from Glaxo Smith Kline. Dr Loewy has received research funding from Genentech. This article’s contents are solely the responsibility of the authors.

References

Schizophrenia Bulletin vol. 38 no. 3 pp. 414 425. 2012 doi:10.1093/schbul/sbr155

Advance Access publication on November 10, 2011

Mobile Assessment and Treatment for Schizophrenia (MATS): A Pilot Trial of An Interactive Text-Messaging Intervention for Medication Adherence, Socialization, and Auditory Hallucinations

Eric Granholm1,2,*, Dror Ben-Zeev3,4, Peter C. Link1, Kristen R. Bradshaw1, and Jason L. Holden2

Psychology Service, VA San Diego Healthcare System, 3350 La Jolla Village Drive, San Diego, CA 92161; 2Department of Psychiatry, University of California San Diego, San Diego, CA; 3Dartmouth Psychiatric Research Center, Dartmouth Medical School, Hanover, NH; 4Thresholds-Dartmouth Research Center, Chicago, IL

*To whom correspondence should be addressed; tel: (858) 552-8585, fax: (858) 642-6416, e-mail: egranholm@ucsd.edu


Mobile Assessment and Treatment for Schizophrenia (MATS) employs ambulatory monitoring methods and cognitive behavioral therapy interventions to assess and improve outcomes in consumers with schizophrenia through mobile phone text messaging. Three MATS interventions were developed to target medication adherence, socialization, and auditory hallucinations. Participants received up to 840 text messages over a 12-week intervention period. Fifty-five consumers with schizophrenia or schizoaffective disorder were enrolled, but 13 consumers with more severe negative symptoms, lower functioning, and lower premorbid IQ did not complete the intervention, despite repeated prompting and training. For completers, the average valid response rate for 216 outcome assessment questions over the 12-week period was 86%, and 86% of phones were returned undamaged. Medication adherence improved significantly, but only for individuals who were living independently. Number of social interactions increased significantly and a significant reduction in severity of hallucinations was found. In addition, the probability of endorsing attitudes that could interfere with improvement in these outcomes was also significantly reduced in MATS. Lab-based assessments of more general symptoms and functioning did not change significantly. This pilot study demonstrated that low-intensity textmessaging interventions like MATS are feasible and effective interventions to improve several important outcomes, especially for higher functioning consumers with schizophrenia.

Key words: mobile interventions/ambulatory monitoring/ experience sampling method (ESM)/ecological momentary assessment (EMA)/schizophrenia/cognitive behavioral therapy/medication adherence/social functioning/auditory hallucinations

Introduction

Ecological momentary assessment (EMA), also called the experience sampling method (ESM), is an ambulatory data collection technique that allows the real-time in vivo assessment of behaviors, moods, thoughts, symptoms, and other daily experiences.1-3 Modern EMA takes advantage of mobile devices such as personal digital assistants and smart phones to signal participants several times throughout the day to respond to questionnaires about their daily lives. EMA provides a temporal accounting of daily experiences that can reveal microprocesses within individuals, such as dynamic relationships between one’s immediate state and subsequent symptoms or impairment in the context of one’s natural environment. Prior research has demonstrated the feasibility and validity of mobile-device EMA methods in consumers with schizophrenia.4-6 Using mobile-device EMA methods in this population, studies have identified important real-time associations between greater anxiety, stress, and arousal and greater severity of psychotic symptoms,3,6-11 negative mood states, and substance abuse12 as well as associations between greater positive affect and better social functioning.13

Mobile devices have also been used to deliver interventions for a variety of health and mental health prob-lems,14,15 but there has been little prior application of mobile interventions in serious mental illness (SMI). Mobile technologies that incorporate EMA methods have great potential for real-time real-world interventions for schizophrenia.16 Mobile devices can be used to deliver services outside the clinic as well as strengthen clinicbased services. For example, daily EMA sampling (eg, of symptoms, moods, medication adherence) via mobile devices could be used to prompt coping responses in consumers reporting increased warning signs or to alert providers to trends in their consumers’ symptoms (eg, escalating severity of hallucinations) to prompt contact and intervention. Mobile devices could also be used to prompt health-promoting behaviors (eg, medication adherence, diet, exercise) or strengthen interventions by prompting in vivo skills practice (homework), which could reduce the intensity of interventions (eg, number or duration of face-to-face sessions). All of these potential uses of mobile-device interventions could also help reduce service costs.

© The Author 2011. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com.


Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


We are aware of only 1 prior study that employed mobile technology to provide an intervention beyond selfmonitoring for consumers with SMI.17 Spaniel and colleagues17 described a 1-year open trial with 45 consumers and 39 family members in which participants were sent a weekly 10-item questionnaire about early warning signs of schizophrenia via text messaging on mobile devices. A threshold number of warning signs triggered an alert to their psychiatrist. The study reported a 60% reduction in the number of hospitalizations relative to the year prior to ambulatory monitoring.17

Given the limited research on mobile-device interventions in SMI, we conducted a pilot trial of a novel mobile phone text message intervention called Mobile Assessment and Treatment for Schizophrenia (MATS).18 The MATS project is the product of collaboration between academic clinical researchers, programming/technology experts, and various stakeholders (ie, program officials, providers, and consumers) from Assertive Community Treatment (ACT) programs in San Diego County. MATS provides mobile interventions to improve high-priority outcomes for stakeholders. Through focus groups, stakeholders recommended 3 treatment targets: Medication adherence, socialization, and auditory hallucinations. The MATS intervention prompted healthpromoting behaviors and used cognitive behavioral therapy (CBT) techniques.19,20 Participants were given mobile phones and received 3 sets of interactive textexchanges each of 6 days per week for 12 weeks. It was hypothesized that significant improvements would be found for each treatment target over the 12-week MATS intervention.

Methods

Participants

Community-dwelling individuals with schizophrenia or schizoaffective disorder (N = 55) over the age of 18 were enrolled (no other inclusion/exclusion criteria). Participants were recruited from outpatient residential and treatment settings (N = 14 from ACT teams) in the Veterans Affairs San Diego Healthcare System and the San Diego County Mental Health System from 2007 to 2010. Participants were not excluded for active substance use disorders, but one participant who reported using methamphetamine on the baseline assessment day was excluded for inability to complete assessments. Diagnoses of schizophrenia included 32 paranoid, 10 undifferentiated, 2 disorganized, and 11 schizoaffective disorders. At baseline, 37 participants were prescribed at least one atypical antipsychotic medication, 23 at least one typical antipsychotic, 31 both typical and atypical antipsychotics, and 1 individual was not prescribed any antipsychotic medications. Twenty-four participants were also prescribed antidepressant medications, and 29 reported using mood-stabilizers.

Procedures

All study procedures were approved by the institutional review board of the University of California, San Diego. Text messages focusing on 3 intervention domains (medication adherence, socialization, and auditory hallucinations) were sent to participants daily from Monday through Saturday for a 12-week period. Following informed consent and baseline assessments, participants were given a basic Motorola cellular phone (model V195 or W490) with full access to domestic calling, calendar, alarm, and gaming functions. Participants were trained to send and receive text messages and typically required one additional in-home visit (about 10 min) for retraining during the initial days of the trial. A PowerPoint presentation was used to describe text-messaging procedures in an initial 30-minute training session in the lab, and participants were guided through approximately 3 practice trials with each of the 3 treatment targets (medications, socializing, and voices). Information about how to use and charge the phone was also provided (eg, how to make outgoing calls, set an alarm, etc.), and participants were given a copy of the PowerPoint presentation to take home. The primary outcome measures were self-reported medication adherence, number of social interactions, and severity of auditory hallucinations obtained through daily ambulatory monitoring (responses to text message questions) over the 12-week intervention period. Secondary outcomes were also assessed using a battery of laboratory-based symptom and functioning measures administered at baseline and the end of the 12-week intervention. Participants received $35 for completing assessment visits and a $20 gift card (Subway or Starbucks) incentive every 2 weeks for completing mobile assessments. All text message responses for each participant were viewable on a secure website, so staff could contact individuals to remind them to answer the text messages or to provide technical support.

Mobile Assessment and Treatment for Schizophrenia

Three sets of 4 text messages (12 total) were sent to participants each day, Monday through Saturday, with each message set targeting 1 of the 3 intervention domains: medication adherence, socialization, or auditory hallucinations. All 3 interventions were delivered in random order each day in the morning, afternoon, and evening. The number and frequency of text messages were based, in part, on focus group feedback and the amount of time needed to send and respond to text messages. Our goal was to adequately sample daily behavior and intervene frequently enough to have a potential impact on the target outcomes, without overburdening individuals with time-consuming messages. Focus group feedback was that more frequent messaging, especially for a 3-month period might be too much. To accommodate daily routines, participants were allowed to choose the specific times they would receive messages within a 2-hour window. Each time a text message was received, the phone generated an auditory signal and/or a vibration that prompted participants to read the message.

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


The text-messaging interventions incorporated CBT techniques.19,20 Thoughts about medications, socializing, and voices were elicited (eg, ‘‘Do you think your voices are powerful?”), and the next messages encouraged participants to question unhelpful beliefs (eg, ‘‘Maybe your voices can’t really do what they say”) and try a behavioral experiment (eg, ‘‘Try ignoring them and see what happens’’). Evidence used to challenge unhelpful beliefs included personalized information provided by the participant. During a baseline interview, the rater who performed the research assessments asked a standard set of questions to elicit personalized information to be used in the text messages. Participants were asked to report at least one benefit of medications and socializing, and a coping strategy that reduced the frequency or distress related to voices (eg, ‘‘What is a benefit or something good about taking your medication?’’; ‘‘What is something you like to do for fun with other people?’’; ‘‘What do you do to help cope with voices?’’). This information was used to create personalized thoughtchallenging messages. For medication adherence, the messages were ‘‘But you said taking meds helped you (personal reported benefit from taking medications).’’ For social functioning, the message was, ‘‘But you said that (personal enjoyable social activity) was fun.’’ For auditory hallucinations, the message was ‘‘You said that (personalized effective coping strategy) helps.’’ The individualized text messages were entered into a secure website, which could only be accessed by research staff. Once the content was entered, the interventions were sent automatically by a remote secure server.

The flow of messaging, including branching according to participant responses, is shown for each intervention domain in figures 1-3. The 4 types of messages sent for each intervention domain were as follows:

If the participant did not reply to the first or second question, the next messages in the sequence were not sent. If the participant reported positive outcomes (eg, they were taking medications or voices were absent or they socialized with 4 or more people), they still received a second question that required a response (eg, ‘‘What’s helping?’’) plus the 2 additional messages suggesting coping strategies. Therefore, there was no reduced hassle or burden for reporting a positive outcome. At maximum compliance, each participant would have received 840 total messages over the 12-week intervention period (4 messages 3 times per day = 12 per day x 6 days per week (10 messages on Friday) = 70 per week x 12 weeks = 840 total messages, with 420 requiring responses). Every Friday, a message was sent to the participants asking them how helpful they found the messages to be that week (1 = Not at all; 2 = Somewhat; 3 = Moderately; and 4 = Very).

Measures

The primary outcome measure for each intervention domain was the daily ambulatory monitoring outcome assessment question for that domain (question 1 described above). Secondary outcomes were also assessed using a battery of laboratory-based symptom and functioning measures. The battery included the Positive and Negative Syndrome Scale (PANSS),21 Beck Depression Inventory—2nd Edition (BDI-II),22 Independent Living Skills Survey (ILSS),23 and American National Adult Reading Test (ANART).24 Interrater reliability was .88 for PANSS total.

Statistical Analyses

Hierarchical generalized linear modeling (HGLM) was used to analyze the text message outcome measures. All models used a multinomial sampling model with a multinomial logit link function. Models were estimated using time (in days) as the lone predictor, except for the model for medication adherence, which also included

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019



living situation as a predictor. Given that individuals in supported living environments are often aided by staff in taking their prescribed medications, a dichotomous

variable that indicated whether the consumer lived independently or in an assisted living facility (independent living = 0.5; assisted living = —0.5) was added as a predictor


Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019



of both the intercept and growth parameters for the medication adherence outcome model. The test of the effect of day on outcome in these models examines whether there is an association between an additional day of treatment and the log-odds of being in one response category relative to the reference category.

Paired samples t tests were used to test for differences between pretreatment and posttreatment on the secondary outcome lab-based measures. To examine potential differences between the sample of consumers used in the analyses and the consumers who withdrew, independent samples t tests were used to test for baseline differences on the lab-based measures. HLM v6.06 was used for the HGLMs; SPSS v11.5 was used for all other statistical analyses; and MATLAB r12.1 was used to produce the HGLM figures.

Results

Of the 55 participants enrolled, 13 were defined as noncompleters because they did not send any valid messages or stopped sending valid messages within 2 weeks, despite repeated trainings and reminders. Independent samples t tests showed several baseline differences between completers and noncompleters (table 1). Noncompleters had lower self-reported living skills (ILSS), more severe negative symptoms (PANSS negative), and lower estimated premorbid verbal IQ (ANART) than completers. No differences in positive symptoms (PANSS positive), depression (BDI-II), age, or education were found between groups. The 42 remaining active participants had a mean age of 48.7 years (SD = 9.1), mean of 12.3 years of education (SD = 1.3) and 69% were male, 74% Caucasian, 7% African American, and 10% Hispanic. Fifty-seven percent of participants resided in assisted living facilities (board and care).

For the 42 completers, the valid response rate over the 12-week intervention period for question 1 for each intervention domain was M = 86%, Median = 93%, SD = 19% for medication adherence; M = 83%, Median = 88%, SD = 19% for socialization; and M = 86%, Median = 94%, SD = 19% for the auditory hallucination intervention. There were 2 possible questions for question 2, depending on how participants responded to question 1 (see figures 1-3). The valid response rates for these 2 questions for each intervention domain were M = 85% and 85%, Median = 94% and 98%, SD = 21% and 30% for medication adherence; M = 78% and 85%, Median = 88% and 90%, SD = 27% and 18% for socialization; and M = 85% and 84%, Median = 94% and 98%, SD = 21% and 30% for the auditory hallucination intervention.

The majority of phones (86%) were returned intact. One phone was never returned, as the participant moved out of state without notice, and 5 phones were damaged or malfunctioned (eg, exposure to water; cracked screen). Participant responses to the question asked each Friday, ‘‘How helpful were the text messages this week?” (1 = Not at all,

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


Table 1. Characteristics of Protocol Completers and Noncompleters

Clinical Measure

Completer (n = 42)

Noncompleter (n = 13)

Statistics

M

SD

M

SD

t

df

P

d

ILSS 10 domain average

0.682

0.105

0.616

0.102

1.99

51

.052

0.65

PANSS total

63.9

18.2

69.3

19.7

0.92

53

.364

0.30

PANSS positive total

17.8

6.4

16.2

6.1

0.78

53

.437

-0.25

PANSS negative total

15.2

6.3

20.7

8.1

2.56

53

.013

0.83

BDI-II total

15.7

12.6

15.8

10.9

0.03

53

.979

-0.01

ANART IQ estimate

103.7

8.6

98.2

7.8

2.04

52

.046

0.66

Age (y)

48.7

9.1

48.9

7.9

0.07

53

.947

-0.02

Education (y)

12.4

1.3

11.8

0.7

1.57

53

.123

0.51

Note: ILSS, Independent Living Skills Survey; PANSS, Positive and Negative Syndrome Scale; BDI, Beck Depression Inventory; ANART, American National Adult Reading Test.


2 = Somewhat, 3 = Moderately, 4 = Very; Mean = 3.15, Median = 3.42, SD = 0.84), suggested that participants found the MATS intervention to be moderately to very helpful. In addition, relative to the ‘‘not at all helpful” response, the odds of reporting all other response categories increased significantly with an additional day (‘‘somewhat,” OR = 1.03, t = 2.33, P = .025; ‘‘moderately,” OR = 1.04, t = 3.35, P = .002; ‘‘very helpful,” OR = 1.04, t = 2.77, P = .009), suggesting greater experience with the intervention increased the likelihood of reporting the intervention was helpful.

Results from the HGLM analysis for the first outcome assessment question from each intervention are presented in table 2. In the model for medication adherence, the effect of an additional day of treatment and the living situation by time interaction were significantly negatively associated with the log-odds of forgetting to take medication, relative to taking medication as prescribed. This suggested that an additional day of treatment aided consumers with remembering to take their medication, and consumers living independently benefited to a greater extent than those in assisted living situations. Figure 4 depicts the probability of endorsing each answer to the medication adherence outcome question each day over the 12-week treatment period. Medication adherence was reported to be high at initial assessment and remained high over time for all participants. Although consumers living independently were initially less likely to report medication adherence than those in assisted living facilities, by the end of treatment, they eventually caught up and even exceeded the adherence rates reported by consumers living in assisted living settings by the end of the 3-month MATS intervention (figure 4, top). The statistically significant intercept terms indicated that at initial assessment, participants were much more likely to report medication adherence rather than forgetting to take medication, not wanting to take medication, or only taking some of their medication. However, participants living independently were less likely to report medication adherence than those in assisted living facilities. Those living independently reported a higher probability of forgetting to take medication at baseline than those in an assisted living setting, but the probability of reporting forgetting diminished over the course of the MATS intervention (figure 4, bottom).

In the model for socialization, the effect of an additional day of treatment was significantly negatively associated with the log-odds of socializing with 1 person, relative to 4 or more. This suggested that an additional day of treatment significantly increased the odds of having 4 or more social interactions relative to having only 1 social interaction. Figure 5 depicts the probability of endorsing each answer to the socialization outcome question each day over the 12-week treatment period. The figure shows that the probability of maximal socialization (4+ interactions) outside of the home increased steadily over the course of the 3-month treatment, while the probability of having only 1 social interaction per day decreased over 10%. There was no significant change in the probability of reporting 0 or 2-3 interactions.

In the model for auditory hallucinations, there was a statistically significant negative association between time and the log-odds of being moderately bothered by hallucinations, relative to having no hallucinations. This suggested that an additional day of treatment significantly increased the odds of reporting not having any voices, relative to being moderately bothered by voices. Figure 6 depicts the probability of endorsing each answer to the auditory hallucination outcome question each day over the 12-week treatment period. The figure shows that the probability of reporting being moderately bothered by hallucinations decreased at a rate similar to the rate at which the probability of reporting no hallucinations increased over the course of treatment. Statistically significant intercept terms indicated that at initial assessment, consumers were more likely to report no voices than extreme or moderate hallucinations (figure 6).

Similar HGLM analyses were also used to examine changes in responses to the second (current cognitions) question during treatment. For medication adherence

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


Table 2. HGLM Results for the Outcome Question of Each Text Intervention


Outcome Domain


Parameter

Estimate


OR


Medication adherence

‘‘Did you take your meds today?” For “forgot” (relative to “yes”)

Intercept

-2.539

0.079

-7.21

<.001

Living situation

1.405

4.076

1.99

.053

Day number

-0.025

0.976

-3.17

.003

Day number x living situation

-0.039

0.962

-2.48

.018

For ‘‘don’t want to” (relative to ‘‘yes”)

Intercept

-4.961

0.007

-7.58

<.001

Living situation

0.400

1.492

0.31

.761

Day number

0.003

1.003

0.21

.835

Day number x living situation

-0.012

0.988

-0.50

.621

For ‘‘only some’’ (relative to ‘‘yes’’)

Intercept

-3.803

0.022

-8.11

<.001

Living situation

1.756

5.788

1.87

.068

Day number

-0.011

0.990

-1.16

.252

Day number x living situation

-0.035

0.966

-1.93

.060

Socialization

‘‘How many people have you socialized with outside the home?’’ For ‘‘no interactions’’ (relative to ‘‘4+’’)

Intercept

-0.256

0.774

-0.95

.349

Day number

-0.002

0.998

-0.48

.633

For ‘‘1 person’’ (relative to ‘‘4+’’)

Intercept

-0.477

0.621

-1.85

.071

Day number

-0.011

0.989

-2.21

.033

For ‘‘2-3 people’’ (relative to ‘‘4+’’)

Intercept

-0.211

0.809

-0.94

.352

Day number

-0.002

0.998

-0.49

.627

Auditory hallucinations

‘‘Have you been bothered by voices?’’

For ‘‘extremely’’ (relative to ‘‘no voices’’)

Intercept

-2.370

0.093

-4.80

<.001

Day number

-0.003

0.997

-0.39

.695

For ‘‘moderately’’ (relative to ‘‘no voices’’)

Intercept

-1.121

0.326

-2.55

.015

Day number

-0.019

0.981

-2.43

.020

For ‘‘a little’’ (relative to ‘‘no voices’’)

Intercept

-0.309

0.734

-0.86

.393

Day number

-0.002

0.998

-0.53

.597


Note: HGLM, hierarchical generalized linear modeling. Separate models for socialization and auditory hallucinations that included living situation (staff assisted vs independent) did not show any significant main effects or interactions involving living situation.


question, ‘‘Do meds help you stay healthy?,” a significant negative association was found between time and the logodds of responding ‘‘Not at all” (OR = 0.96, t = -3.33, P = .002) or ‘‘Not sure” (OR = 0.96, t = —2.66, P = .012), relative to the ‘‘Yes definitely” response category. This suggests participants changed from believing medications were not helpful to believing medications help them stay healthy. For the socialization intervention, a significant negative association was found between time and the log-odds of responding that socializing is a ‘‘waste of time” (OR = 0.98, t = —2.27, P = .029) and “dangerous” (OR = 0.98, t = —2.79, P = .009), relative to the “other”


response category. This suggests a reduction in conviction in these negative beliefs about socializing. For the auditory hallucinations intervention, a significant negative association was found between time and the log-odds of reporting voices are “uncontrollable” (OR = 0.98, t = —2.28, P = .028) and marginally for ‘‘all knowing” (OR = 0.98, t = —1.95, P = .058), relative to the “other” category. This suggests a reduction in conviction in the beliefs that voices are uncontrollable or all knowing.

With regard to secondary outcomes, paired samples t tests showed no significant differences between baseline and posttreatment assessments for any of the laboratory


Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019



Medication Adherence Intervention: Question #1


Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019



assessments of symptoms (PANSS, BDI-II) or functioning (ILSS) (ds = —0.14-0.05). Because these scales measured global symptom domains, we explored pre-post differences on individual PANSS items that more specifically indexed the target outcomes. However, the pre-post difference on the PANSS hallucinations item (P3) also was not significant, Mpre = 3.67, SD = 1.76, Mpost = 3.60, SD = 1.74, t41 = 0.29, P = .867, d = 0.04, and the pre-post difference for the PANSS social withdrawal item (N4), was marginally significant, Mpre = 3.05, SD = 1.64, Mpost = 2.54, SD = 1.65, t41 = 1.85, P = .072, d = 0.31.

Discussion

This pilot trial of the MATS intervention demonstrated that interactive text message assessments and interventions are feasible for many consumers with schizophrenia. Notably, the average valid response rate for the 3 intervention domains was 83%-86% for question 1 for participants who completed more than 2 weeks of the MATS intervention, and the vast majority of phones (86%) were returned at the end of the trial without incident or damage. The MATS intervention was also effective at improving medication adherence, socialization, and auditory hallucinations for many consumers.

With regard to medication adherence, the MATS intervention was most effective for individuals who were living independently. These consumers likely benefited most from MATS because, unlike those residing in supported living environments, they had fewer supports in place to assist in taking medications. The improvement in reporting medication adherence was relative to reporting ‘‘forgot,'' which suggests the text messages may have served as a reminder to take daily medications, especially for

Fig. 6. Hierarchical generalized linear modeling-derived trajectories of responses to ‘‘Have you been bothered by voices?’’.

consumers who did not live in a setting with staff who would remind them. This finding suggests mobile interventions and other behavioral interventions that incorporate routines and other natural prompts in daily life to remind consumers to take medications may improve medication adherence in this population. Over the course of treatment, consumers were also less likely to report that medications do not help them ‘‘stay healthy.’’ This may suggest that the intervention was associated with a reduction in negative beliefs about medications, which may have also contributed to improved adherence.

MATS was associated with improvement in socialization. The probability of maximal socialization (4+ interactions) outside of the home increased steadily over the course of the 3-month treatment, while the probability of having only 1 social interaction per day decreased over 10%. A significant reduction over the trial period was also found in the probability of reporting socializing is ‘‘waste of time’’ or ‘‘dangerous.’’ It is possible that these changes in negative cognitions about social engagement positively impacted day-to-day socialization behavior. Behavioral suggestions (eg, ‘‘Try asking a friend to go for a walk'') may have also prompted interactions with others. The probability of reporting zero social interactions did not change during the MATS intervention.

MATS was also associated with a reduction in severity of auditory hallucinations. The probability of reporting moderate severity of auditory hallucinations decreased, and the rate of increase in reporting no hallucinations increased. In contrast, the probability of reporting extreme hallucinations was rare and did not change significantly over the course of the MATS intervention. The probability of endorsing thoughts that hallucinations are ‘‘uncontrollable'' or ‘‘all knowing'' also decreased significantly during MATS. The thought-challenging and behavioral experiment interventions in MATS may have contributed to reduced conviction in these beliefs about voices, and this reduction in negative beliefs may have been associated with a reduction in hallucinations. It is also possible that the improvements found in medication adherence, rather than changes in cognitions, lead to the reduction in hallucinations, but medication adherence only improved for consumers living independently, whereas improvement in hallucinations was comparable for consumers living independently and living in assisted housing (table 2).

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


The version of MATS used in this pilot study may not be helpful to all consumers with schizophrenia. MATS was delivered on an older generation Motorola phone platform, which was clearly challenging to navigate for some consumers with schizophrenia. Lower functioning consumers with more severe negative symptoms and lower estimated premorbid IQ did not complete the intervention, despite multiple reminders and training sessions. These participants were less successful in mastering the use of these older generation phones, but as discussed below, mobile interventions may be more accessible to these participants with a newer smart phone touch-screen platform that is easier to navigate. More intensive individual or group CBT interventions (eg, a standard course of face-to-face sessions or a combination of MATS and some therapy sessions) may be needed for these consumers.

No significant differences in lab-based secondary outcome measures were found between baseline and end of treatment assessments, which suggests that the benefits of this brief 12-week low-intensity MATS intervention did not generalize to these broader outcomes during the limited pilot study period. The discrepancy between the findings for the lab-based measures and the ambulatory monitory outcome measures may also suggest that the daily ambulatory self-reports were not valid assessments of symptom status and functioning. However, these labbased measures were aggregated global measures of psychopathology and functioning, not specific measures of the MATS treatment targets, like the ambulatory monitoring outcome questions. The pre-post difference on the PANSS hallucinations item (P3) and PANSS social withdrawal item (N4) also were not significant. These item analyses more closely index the target outcomes than the global scores, but they still do not measure the target outcome with the same precision as the EMA text message questions. For example, the PANSS hallucinations item includes auditory, visual, olfactory, and tactile hallucinations, not just the auditory hallucinations targeted in this study and indexed by the text message question. In addition, ESM has been extensively used and validated in consumers with schizophrenia.3,4,6-11 The ambulatory reports provided in this study also were systematically associated with factors that would be predictably associated with the target outcomes (eg, consumers in assisted living situations reported significantly higher medication adherence than consumers living independently). It is possible that ambulatory monitoring is more sensitive to change in specific treatment targets than global retrospective summaries provided in the context of lab-based assessments, which may be less accurate representations of day-to-day symptom, mood, and functioning behaviors.1,2 This topic of disparities between momentary self-reports generated in naturalistic settings and retrospective lab summaries of the same period of time is addressed more extensively in a separate article in the current special issue.

Lessons Learned

In addition to the study findings, the development and pilot testing of the MATS intervention provided valuable experiences to inform future development of ambulatory interventions for consumers with severe mental illness. In this pilot study, the older generation Motorola phone required more steps and manipulation for each action, which proved too difficult for some participants to master. For instance, the backlight of the phone screen remained on for a maximum of 60 seconds, so if a participant required more time to read a question, the screen went blank. Participants described pressing a random button to return to the screen, which would result in a completely different message or phone setting, leading to frustration and lack of response. Some consumers were also confused when they missed previous text messages and found multiple unopened questions in their inbox at one time. Others were confused by the phone configuration, when they were brought back to the previous screen after responding to a text message. These problems prompted some consumers to send multiple responses to the multiple questions or to believe that they had not accurately replied, so they sent additional text message answers to the same question. Multiple responses were coded as invalid.

These problems can be avoided by configuring the phone to delete messages after a response is made or when they are not opened within a specific time period. Using more advanced “smart” phones may also help improve response rates in cellular phone-based assessments and interventions. With “smart” phones, applications could be developed with longer response windows and ease of touch-screen navigation and responding, and once an application was initiated, participants would be unable to accidentally exit it until completion of the survey. A platform that is easier for consumers to navigate may improve the adherence rates for lower functioning consumers who struggled with the MATS intervention in this pilot study.

This study had several limitations. First, medication adherence findings should be interpreted cautiously because consumers tend to inflate their self-reports of medication adherence, when compared with more objective measures, such as pill counts.25 Future studies should include more objective measures of medication adherence, in addition to EMA self-reports. Second, consumers reported high medication adherence, low severity of voices, and multiple social interactions at baseline. Even, greater improvements in these outcomes might have been observed had consumers been selected for nonadherence, severe voices, or social isolation at baseline. Third, we suggested that the change in specific beliefs about medications, socialization, and voices found in this pilot study contributed to change in these outcomes, but it is also possible that change in the outcomes contributed to change in the beliefs. Time-lagged analyses could be used to examine the causal direction of these relationships in future mobile interventions research using larger samples with greater frequency of specific beliefs. Fourth, consumers received gift card incentives for responding to text messages, which may not be feasible outside of a research study. Future research is needed to determine textmessaging response rates and MATS efficacy without such incentives. Finally, the study also lacked a comparison group, so the improvements found could be attributed to standard care, rather than MATS.

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


In conclusion, this pilot study demonstrated that relatively long-term use of mobile technologies to assist in the assessment and treatment of people with serious mental illness is feasible and holds great potential. As cellular phones incorporate more sophisticated technologies, they will likely become more intuitive, affordable, and widespread in use. These preliminary findings for the MATS intervention were encouraging and suggest that, with further development and validation, mobile technologies might facilitate more naturalistic interventions outside of the clinical setting.

Funding

National Institute of Mental Health (P30 MH80002) through the Advanced Center for Innovations in Services and Interventions Research (PI: Dilip Jeste, M.D.) at the University of California, San Diego.

Acknowledgments

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. The Authors have declared that there are no conflicts of interest in relation to the subject of this study. We would like to thank Dr Kevin Patrick and Fred Rabb for programming and cellular phone assistance, and the clinical and administrative staff at Telecare, San Diego.

References

Downloaded from https://academic.oup.eom/schizophreniabulletin/article-abstract/38/3/414/1866511 by guest on 15 July 2019


Original Paper

Mobile-Assisted Cognitive Behavioral Therapy for Negative Symptoms: Open Single-Arm Trial With Schizophrenia Patients

Eric Granholm1, PhD; Jason Holden2, PhD; Kristen Dwyer2, PhD; Tanya Mikhael2, BA; Peter Link2, MA; Colin Depp12, PhD

1VA San Diego Healthcare System, San Diego, CA, United States

2Department of Psychiatry, University of California, San Diego, San Diego, CA, United States

Corresponding Author:

Eric Granholm, PhD

VA San Diego Healthcare System

9500 Gilman Drive #0737

San Diego, CA

United States

Phone: 1 858 534 2542

Email: egranholm@ucsd.edu

Abstract

Background: Negative symptoms are an important unmet treatment need for schizophrenia. This study is a preliminary, open, single-arm trial of a novel hybrid intervention called mobile-assisted cognitive behavioral therapy for negative symptoms (mCBTn). Objective: The primary aim was to test whether mCBTn was feasible and could reduce severity of the target mechanism, defeatist performance attitudes, which are associated with experiential negative symptoms and poor functioning in schizophrenia. Methods: Participants with schizophrenia or schizoaffective disorder (N=31) who met prospective criteria for persistent negative symptoms were enrolled. The blended intervention combines weekly in-person group therapy with a smartphone app called CBT2go. The app extended therapy group skills, including recovery goal setting, thought challenging, scheduling of pleasurable activities and social interactions, and pleasure-savoring interventions to modify defeatist attitudes and improve experiential negative symptoms.

Results: Retention was excellent (87% at 18 weeks), and severity of defeatist attitudes and experiential negative symptoms declined significantly in the mCBTn intervention with large effect sizes.

Conclusions: The findings suggest that mCBTn is a feasible and potentially effective treatment for experiential negative symptoms, if confirmed in a larger randomized controlled trial. The findings also provide support for the defeatist attitude model of experiential negative symptoms and suggest that blended technology-supported interventions such as mCBTn can strengthen and shorten intensive psychosocial interventions for schizophrenia.

Trial Registration: ClinicalTrials.gov NCT03179696; https://clinicaltrials.gov/ct2/show/NCT03179696

(JMIRMentHealth 2020;7(12):e24406) doi: 10.2196/24406

KEYWORDS

motivation; persistent negative symptoms; dysfunctional attitudes; mHealth; blended intervention; mobile phone

Introduction

Negative symptoms account for much of the poor functional outcome in schizophrenia and are an unmet treatment need [1-3]. Negative symptoms can refer to reduced expressive (eg, facial affect and voice tone) or experiential (eg, avolition and asociality) symptoms, which comprise two separate factors [4-6]. Experiential negative symptoms are particularly important to treat, because they are strongly associated with functioning [5,7]. Unfortunately, available pharmacological treatments have only limited benefits for negative symptoms [8,9].

Beck and colleagues [10-12] have proposed that interventions that reduce defeatist attitudes may improve negative symptoms and functioning in schizophrenia. Several studies have found that cognitions such as defeatist performance (eg, “Why try, I always fail”) and social disinterest (eg, “I’m better off alone”) attitudes are associated with negative symptoms, and to some extent poor functioning, even after accounting for depression [10,13-18]. In social learning theory [19], self-competency beliefs are also central to motivation for achievement and engagement in effortful goal-directed activities. The Beck model hypothesis is that defeatist attitudes lead to low motivation and avoidance of effortful goal-directed activities. Thus, an intervention that targets defeatist attitudes may increase motivation and effort toward goal-directed activities.

https://mental.jmir.org/2020/12/e24406

XSL-FO


Defeatist attitudes can be targeted in cognitive behavioral therapy (CBT). Clinical trials of CBT for psychosis have found mixed results for reducing negative symptoms [20,21], but some CBT interventions that specifically targeted defeatist attitudes have found more promising results [22-27]. Social skills training (SST) has also produced significant but modest improvements in negative symptoms [28,29]. In our cognitive behavioral social skills training (CBSST) intervention, which combines CBT and SST [30], we have found significant improvements in defeatist attitudes and in experiential negative symptoms and functioning; improvements in experiential negative symptoms were mediated by improvements in defeatist attitudes [23]. Modification of defeatist attitudes, therefore, may be a mechanism of change in CBSST, whereby reduction in defeatist attitudes contributes to increased motivation and effort toward goal-directed tasks. CBSST, however, is an intensive and lengthy (ie, up to 36 sessions), high-burden, multicomponent intervention. If the intervention could be shortened and the focus on reducing defeatist attitudes strengthened by using mobile interventions, the cost and burden of CBSST implementation could be reduced. In this way, blended interventions that combine mobile interventions with in-person interventions could increase access to evidence-based interventions for schizophrenia by reducing implementation barriers.

Smartphones are widely available, affordable, and frequently used by individuals with serious mental illness [31-34]. In previous clinical trials, we have found significant improvements in social functioning and symptoms in individuals with schizophrenia and bipolar disorder using 12-week mobile interventions that incorporate CBT principles with minimal therapist contact [35,36]. A number of other mobile interventions for schizophrenia have been developed and received preliminary testing, but few have specifically targeted negative symptoms or motivation and few have blended in-person plus app interventions; for reviews, see Camacho et al [37] and Firth and Torous [38]. One recent pilot trial of a blended intervention for psychosis [39] used a brief coping-focused intervention for distressing voices—SAVVy (Smartphone-Assisted coping-focused interVention for Voices)—which blended four sessions of in-person therapy with ecological momentary assessment (EMA) or ecological momentary intervention between sessions; the trial found that the intervention was feasible, improved coping, and, at a trend level, reduced severity of voices. Another app-only trial that did target motivation in schizophrenia—PRIME (Personalized Real-time Intervention for Motivational Enhancement) [40,41]—led to improvements in self-efficacy beliefs and social interactions with peers, as well as reduced defeatist beliefs and self-reported negative symptoms in individuals with recent-onset schizophrenia. These trials support the feasibility and potential benefits of this approach.

Given the promise of in-person and mobile CBT interventions targeting defeatist attitudes and motivation in schizophrenia, we developed a blended intervention, called mobile-assisted cognitive behavioral therapy for negative symptoms (mCBTn), which combines in-person, 90-minute, weekly groups with a mobile app called CBT2go. The mCBTn intervention primarily targets defeatist attitudes to improve experiential negative symptoms in schizophrenia. The CBT components and skills-training approach of our CBSST group intervention were combined with mobile thought-challenging interventions that were based on our Mobile Assessment and Treatment of Schizophrenia (MATS) [35] and CBT2go [36] interventions. This thought-challenging algorithm incorporated EMA to sample attitudes and moods in real-world contexts and then used personalized evidence to challenge dysfunctional beliefs (eg, if a participant rates their expectation for pleasure in a planned social interaction at a clubhouse as low, they would receive the challenge, “But you said you had fun at the clubhouse last time”). In addition to thought challenging, the CBT2go app and group intervention used in mCBTn both also incorporated recovery goal setting and tracking, scheduling pleasurable activities and social interactions, and pleasure-savoring interventions.

We conducted an open preliminary trial of mCBTn in patients with schizophrenia or schizoaffective disorder with moderate to severe persistent negative symptoms, and hypothesized that defeatist attitudes and experiential negative symptoms would be significantly reduced from baseline to end of treatment. This open trial was funded as part of the National Institute of Mental Health Experimental Therapeutics Program (RFA-MH-18-704 R61/R33), which involves preliminary testing of an intervention's impact on a target mechanism (ie, defeatist attitudes) associated with an important clinical outcome (ie, experiential negative symptoms). The primary aim of the study was target engagement; that is, we hypothesized that mCBTn would lead to a significant reduction in severity of defeatist attitudes. We also assessed participants at 12, 18, and 24 weeks of treatment to determine which dose of treatment could produce at least a medium effect size (Cohen J=0.5) improvement in defeatist attitudes. In the Experimental Therapeutics Program, contingent on changing the target in this single-arm open-trial phase, a larger randomized controlled trial (RCT) will be conducted with treatment at the dose identified.

Methods

This was a single-arm, open-trial, pre-post evaluation of the feasibility and preliminary effect of mCBTn. This trial was registered at ClinicalTrials.gov (NCT03179696).

Sample

The study protocol was reviewed and approved by the Institutional Review Board of the University of California, San Diego, prior to initiating research activities with participants. Participants with schizophrenia or schizoaffective disorder were selected who have moderate to severe persistent experiential negative symptoms [3], as well as moderate to severe defeatist attitudes, recognizing that an intervention targeting defeatist attitudes is not likely to be helpful for consumers who do not

have them. Participants were required to meet all inclusion and exclusion criteria over a 2-week evaluation phase. Inclusion criteria were as follows:

Exclusion criteria were as follows:

Blended mCBTn Intervention

The therapy group and the mobile app integrated skills-based interventions, including recovery goal setting, thought challenging, scheduling of pleasurable activities and social interactions, and pleasure-savoring interventions to modify defeatist attitudes and improve motivation and pleasure negative symptoms. A modified 12-session version of the Cognitive Skills Module of CBSST [30] was delivered in 90-minute, weekly group therapy sessions with two masters-level therapists and approximately 6 participants per group. The 12-session module provided all the core mCBTn content within the minimum number of sessions that might be identified as the optimal dose, but the module was repeated for a total of 24 sessions to determine if more sessions were needed to change the defeatist attitude target.

The mCBTn manual included a therapist guide and a patient workbook describing the skills and homework assignments, as well as a collection of games and exercises to make learning fun and promote engagement [30]. The module began with setting a meaningful living, learning, working, or socializing recovery goal, and the goal was broken down into short-term goals and goal steps. Defeatist attitudes and avoidance behaviors that interfere with working on the goal were then modified using CBT skills. Group members were introduced to the general concepts of CBT, including the relationship between thoughts, actions, and feelings (ie, generic cognitive model); thought challenging through behavioral experiments and examining evidence for beliefs; and mistakes in thinking. The primary thought-challenging skill trained was the 3Cs: Catch It, Check It, Change It—“It” is an unhelpful defeatist thought. Group time was also spent practicing the smartphone interventions, developing content for the mobile device (eg, personalized motivational and thought-challenging statements about goals, socializing, and pleasurable activities), and reviewing data collected with the device to challenge defeatist attitudes.

An iPhone 5s or 5SE was provided to all participants with an unlimited data plan and could receive and send unlimited texts and phone calls. Mobile interactions were triggered by an app notification in the morning, with reminder prompts at midday and evening. If participants responded to the notification earlier in the day, the second and/or third notifications were not delivered. The purpose of the notifications was to prompt daily engagement with the app. Device training was provided on how to operate and charge the device, the meaning of all questions and response choices, procedures for carrying the device, responding to prompts, how to access crisis lines, and how to use various apps. This information was also provided in a written manual given to participants. Participants returned the device at the end of treatment.

The CBT2go app was used to prompt and track each group member's goal-directed activities in the community, facilitate adherence to homework assignments involving community practice of thought-challenging skills trained in group, and prompt performance and savoring of personalized pleasurable activities and social interactions planned in group. The CBT2go app used personalized statements developed in group to challenge social disinterest and defeatist attitudes in real-time, real-world environments. After groups, therapists could enter personalized comments that participants made in group into a web-based dashboard (eg, “Having coffee with Jim is fun” and “Angie always makes you laugh”), which were used by the app to challenge low expectation ratings of motivation, anticipatory pleasure, or anticipated success for planned activities (see sample screenshots in Figure 1).

Participants carried the device and received the mobile intervention for the entire 24-week, blended group-plus-app intervention period. Each day during treatment, the CBT2go app alerted participants in the morning to make an action plan, which involved responding to a multiple-choice question about what to do that day: (1) a goal step, (2) social interaction, (3) pleasurable activity, or (4) homework assignment, or take the day off. Following the participant's choice, the CBT2go app asked for three EMA ratings (slide bar ranging from 1 [not at 

https://mental.jmir.org/2020/12/e24406

XSL-FO


all] to 7 [very much]) of (1) motivation to do the activity, (2) anticipated success, and (3) anticipated pleasure for the action plan. High ratings were reinforced (eg, “That's right, socializing can be fun”), and low ratings were challenged (eg, “Don't forget, you said walking on the beach was fun” or “But your goal is to make a close friend”). Behavioral experiments were then suggested to test out beliefs (eg, “Try asking someone to go for a short walk”). If a pleasurable activity action plan was selected, the pleasurable activities component of the CBT2go app was opened, which displayed several personalized activities that the participant previously entered. For the selected activity, the time and place to do the activity was queried (ie, a reminder alert was delivered at the time planned) and anticipated pleasure ratings were queried on a slide bar ranging from 0 (low) to 10 (high). Pleasure was again rated if the activity was completed, and these ratings were saved and available on demand in the group sessions for the therapist to discuss how people often think activities will not be as fun as they turn out to be and how this low expectation can reduce the likelihood of doing the activity. The app also prompts pleasure savoring of completed activities by taking a selfie or other photo or journaling about their experiences in the app. These photos and journal entries are also available on demand for review by participants or therapists in group sessions. Therapists helped develop the plans for pleasurable activities in group. Finally, if homework was the action plan selected, participants were directed to use their homework sheets and workbooks from the group to complete their weekly homework assignment.

Finally, an on-demand recovery goal-setting component of the CBT2go app was also provided and was populated during the goal-setting sessions in group by therapists and participants, as well as between sessions for homework by participants. A long-term goal was set and short-term goals and goal steps that would facilitate achievement of the long-term goal were entered into the app. The app could be accessed on demand to remind participants of goals, and goal steps could be checked off to track and motivate goal progress.


Assessments

Participants were assessed at 2 weeks prior to baseline; at baseline; and at 12, 18, and 24 weeks of treatment. Dysfunctional attitudes were measured on the DPAS [10] (ie, primary target mechanism) and the Asocial Beliefs Scale (ABS) [45]. The CAINS-MAP [5] subscale was the primary negative symptom outcome measure. The CDS [43] and the PANSS [42] positive symptom subscale were used to assess secondary symptom outcome domains. Functioning was assessed on the Abbreviated Quality of Life Scale (A-QLS) [46] and the Social Functioning Scale (SFS) [47].

Statistical Analyses

Mixed-effects regression models, utilizing HLM (hierarchical linear modeling) v6.08 (Scientific Software International), were estimated to predict each in-lab outcome assessment and mobile CBT2go app ratings of motivation, success, and anticipated pleasure for activities using time in weeks since baseline as a level-1 predictor. Paired-sample, 2-tailed t tests between baseline and each follow-up assessment tested whether significant change was found for each outcome at each assessment point to inform the dose of treatment that might achieve a significant improvement in DPAS and CAINS-MAP scores with at least a medium effect size (Cohen d=0.5).

Results

Sample

We recruited and assessed 67 participants; see the CONSORT (Consolidated Standards of Reporting Trials) diagram in Figure 2. However, 36 of the 67 participants (54%) were excluded because they did not meet persistent negative symptom criteria. For the 31 participants enrolled in treatment, excellent retention was found with 28 (90%), 27 (87%), and 25 (81%) participants assessed at the 12-, 18-, and 24-week assessment points, respectively. There was only one adverse event, which was a hospitalization for symptom exacerbation in the context of medication nonadherence. The participant remained in the study. The sample had a mean age of 48.3 (SD 9.5) years and a mean of 11.8 (SD 1.5) years of education. The sample was 65% (20/31) male, was 65% (20/31) White, and had moderate baseline symptom severity—the PANSS total mean score was

Figure 2. CONSORT (Consolidated Standards of Reporting Trials) flow diagram of participants through the open trial. CBT: cognitive behavioral therapy; DPAS: Defeatist Performance Attitude Scale.



Excluded (it = 90)

Not interested (n=44)

Lost Conlact/Moved (n=8) Met Exclusion Criteria (n-38)*

Previous CBT (n=5: 13%) Positive Symptoms (n=14i 37%) Substance Use (n=4:11%)

Low Negative Symptoms (n=5; 13%)

Unstable Medications (n=5: 13%) Wrong Diagnosis (n=3:8%) Age (n= 1; 3%)

Language (n=l; 3%)

*Docs not sum to 100% because 2 participants were exchided for more




Excluded (w = 36)

Lost Interest (n=9)

Mel Exclusion Criteria (n=27)*

Positive Symptoms (n=12: +4%)

Low DPAS <n=l 11 41%)

Substance Use(n=5: 19%)

High Depression (n=2i 7%)

Extrapy ramidal Symptoms (n=2i 7%)

Low Negative Symptoms (n=3: 11%)

Wrong Diagnosis (n=2; 7%)

Unable to assess (n=l; 4%) *Does not sum to 100% because 11




Follow Up

Week 12 Assessment (n=28; 90%)

Loss of interest (n=3)

Week 18 Assessment (n=27; 87%)

Loss of interest (n=4)

Week 24 Assessment (n=25; 81%)

Loss of interest (n=5)

Table 1. Outcome variables and paired t tests at each assessment point relative to baseline.

Outcome measure and time points

n (%)

Score, mean (SD)

Cohen da

2-tailed t test (df)

P value

Defeatist Performance Attitude Scale

Baseline

31(100)

66.3 (14.4)

N/Ab

N/A

N/A

12 Weeks

28(90)

60.7 (15.9)

0.40

2.23 (27)

.03

18 Weeks

27 (87)

56.4 (15.7)

0.70

4.02 (26)

<.001

24 Weeks

25(81)

52.2 (17.3)

1.00

4.10 (24)

<.001

Asocial Beliefs Scale

Baseline

30(100)

5.9 (3.1)

N/A

N/A

N/A

12 Weeks

28 (93)

6.5 (2.9)

-0.20

0.58 (27)

.57

18 Weeks

27 (90)

6.2 (3.2)

-0.10

0.21 (26)

.84

24 Weeks

25 (83)

6.4 (3.3)

-0.15

0.19 (24)

.85

Clinical Assessment Interview for Negative Symptoms Motivation and Pleasure

Baseline

31 (100)

23.1 (3.4)

N/A

N/A

N/A

12 Weeks

28 (90)

21.3 (6.1)

0.55

2.07 (27)

.048

18 Weeks

27 (87)

20.5 (6.2)

0.75

2.95 (26)

.007

24 Weeks

25 (81)

20.0 (6.8)

0.90

3.16 (24)

.004

Clinical Assessment Interview for Negative Symptoms Expression

Baseline

31 (100)

4.9 (3.2)

N/A

N/A

N/A

12 Weeks

28 (90)

4.7 (3.6)

0.05

0.56 (27)

.58

18 Weeks

27 (87)

4.3 (3.3)

0.20

1.18 (26)

.25

24 Weeks

25 (81)

4.4 (3.3)

0.15

1.11 (24)

.28

Positive and Negative Syndrome Scale positive symptoms

Baseline

31 (100)

13.4 (4.5)

N/A

N/A

N/A

12 Weeks

28 (90)

13.0 (4.3)

0.10

0.75 (27)

.46

18 Weeks

26 (84)

12.8 (4.8)

0.10

0.64 (25)

.52

24 Weeks

24 (77)

11.5 (4.0)

0.45

2.46 (23)

.02

Calgary Depression Scale

Baseline

31 (100)

3.4 (2.2)

N/A

N/A

N/A

12 Weeks

28 (90)

3.4 (2.6)

0

0 (27)

>.99

18 Weeks

27 (87)

3.1 (2.5)

0.10

0.85 (26)

.40

24 Weeks

25 (81)

2.7 (2.5)

0.30

1.99 (24)

.06

Abbreviated Quality of Life Scale

Baseline

31 (100)

23.3 (6.1)

N/A

N/A

N/A

12 Weeks

28 (90)

25.8 (7.9)

0.40

2.17 (27)

.04

18 Weeks

27 (87)

24.9 (8.3)

0.25

1.78 (26)

.09

24 Weeks

25 (81)

24.8 (8.8)

0.25

1.77 (24)

.09

Social Functioning Scale

Baseline

31 (100)

114.7 (21.8)

N/A

N/A

N/A

12 Weeks

27 (87)

118.5 (22.2)

0.15

2.27 (26)

.03

18 Weeks

27 (87)

115.9 (23.6)

0.05

1.70 (26)

.10

24 Weeks

25 (81)

117.3 (26.7)

0.10

1.70 (24)

.10

aA positive Cohen d value indicates improvement on the outcome from baseline to follow-up.

https://mental.jmir.org/2020/12/e24406

XSL-FO

bN/A: not applicable.

Dysfunctional Attitudes

The effect of time was significant for defeatist performance attitudes (DPAS: y=-0.59, t30=-4.27, P<.001). Change in DPAS score from baseline (see Table 1) was significant at all assessment points during treatment, with medium to large effect sizes. The minimal treatment dose needed to achieve at least a medium effect size was at 18 weeks. In contrast, the effect of time was not significant for asocial beliefs (ABS: y=0.01, t29=0.47, P=.64).

Symptoms

Significant reduction in severity of experiential negative symptoms was found (CAINS-MAP: y=-0.14, t30=-3.12, P=.004) with medium to large effect sizes (see Table 1). The minimal treatment dose needed to achieve at least a medium effect size was at 12 weeks. For expressive negative symptoms, assessed using the CAINS Expression (CAINS-EXP) subscale, the effect of time was not statistically significant and no significant reduction from baseline was found at any assessment point (CAINS-EXP: y=-0.02, t30=-1.06, P=.30). Significant reduction in severity of positive symptoms was found by week 24, but not at earlier assessment points, and the effect of time was not significant (PANSS-positive subscale: y=-0.06, t30=-1.46, P=.16). Similarly, significant reduction in severity of depressive symptoms was found by week 24 but not at earlier assessment points, and the effect of time was not significant (CDS: y=-0.03, t30=-1.66, P=.11).

Mobile Symptom and Attitude Ratings

CBT2go app ratings of motivation and anticipated pleasure and success for completing planned activities increased significantly for motivation (y=0.007, t28=2.17, P=.04) and anticipated pleasure (y=0.008, t28=2.53, P=.02) but not for anticipated success (y=0.006, t28=1.62, P=.12). This indicates steady gains in self-reported motivation and pleasure (0.7-0.8 points per 100 days on a 0-7 scale) over treatment. To estimate effect sizes for these changes over the course of treatment, we computed the mean ratings for the first 4 weeks and last 4 weeks of treatment for mobile CBT2go app ratings. Large increases in mean ratings were found between the first 4 weeks of treatment and the last 4 weeks for motivation (Cohen d=0.65; first 4 weeks: mean 2.9, SD 1.8; last 4 weeks: mean 4.1, SD 1.8) and anticipated pleasure (Cohen d=0.80; first 4 weeks: mean 3.0, SD 1.5; last 4 weeks: mean 4.2, SD 2.1). Medium increases were found for anticipated success (Cohen d=0.55; first 4 weeks: mean 3.1, SD 1.5; last 4 weeks: mean 3.9, SD 1.9).

Functioning

Significant improvement in A-QLS scores was found between baseline and 12 weeks but not at other assessment points, and the effect of time was not significant (y=0.08, t30=1.49, P=.15). Participants also showed significant improvement on the SFS total score between baseline and 12 weeks but not at the other two assessment points, and the effect of time was only at a trend level (y=0.19, t30=1.77, P=.09).

App Engagement

The CBT2go app prompted participants to select an action plan each day during up to 168 days of treatment. There was a mean of 18.7 (SD 21.3) responses to 84 action plan prompts (22%) at 12 weeks and a mean of 32.3 (SD 31.5) responses to 168 action plan prompts (19.2%) at 24 weeks for participants who did not drop out of treatment by each assessment point; this indicates engagement in homework and skills practice more than once per week, with minimal fatigue effects over the course of treatment. The number of action plans completed was not significantly correlated with any symptom measure at baseline (range of r=-0.05 to 0.07).

In HLM analyses examining the association between app engagement and outcome, the number of action plans by time interaction was significant for change in CAINS-MAP scores (Y=-0.003, t29=-2.11, P=.04) but not for DPAS scores (Y=-0.002, t29=-0.36, P=.72). The change in CAINS-MAP scores relative to baseline was also marginally correlated with the number of action plans completed at 12 weeks (r=-0.33, P=.09) and 18 weeks (r=-0.35, P=.07) but not 24 weeks (r=-0.28, P=.18). This did not hold true for change in DPAS scores relative to baseline at 12 weeks (r=-0.23, P=.24), 18 weeks (r=-0.13, P=.51), and 24 weeks (r=-0.07, P=.75). Thus, greater engagement with the app was associated with greater reduction in motivation and pleasure negative symptoms.

Discussion

The results of this open trial of mCBTn showed significant, large, within-group improvements in defeatist attitudes and negative symptoms and defeatist attitudes by 18 weeks of treatment, which demonstrates feasibility and engagement of the defeatist attitudes target, and justifies a larger RCT. These findings also provide support for the defeatist attitude model of negative symptoms [10,12]. Some specificity was found for improvements in defeatist performance beliefs but not asocial beliefs, perhaps because the intervention focused on all living, learning, working, and socializing goals, rather than only socialization. The intensive focus of mCBTn on modifying defeatist beliefs, both in group therapy and in real-world environments using the CBT2go app, resulted in a large improvement in experiential negative symptoms within a relatively short 18-week treatment duration and medium improvement by 12 weeks, which is relatively rapid change. If blended interventions such as mCBTn can strengthen and thereby shorten intensive psychosocial interventions, then implementation barriers associated with burden and cost of lengthy psychotherapy interventions would be reduced. This would improve access to evidence-based practices for consumers with severe and persistent negative symptoms, which would have considerable public health significance. However, further testing in a larger RCT with a control condition is needed to confirm the efficacy of mCBTn and justify work on implementation.

This study adds to the growing literature on CBT-based mobile interventions for schizophrenia [37,38,48,49] and indicates that patients with severe and persistent negative symptoms may be amenable to this approach. In the first trial to use mobile CBT-informed interventions for schizophrenia in a text-messaging platform, we [35] used a similar approach that integrated EMA of symptoms and behaviors (eg, social isolation, medication adherence, and voices) in everyday contexts and delivered personalized just-in-time thought-challenging messages when participants reported symptom distress or defeatist attitudes; we also found improvements in auditory hallucinations, socialization, and medication adherence. In another prior trial using this same algorithm in an earlier version of the CBT2go app, we found improvements in total symptoms in a large RCT of patients with schizophrenia and bipolar disorder [36]. Ben-Zeev and colleagues [50,51] also used a similar CBT-informed algorithm with the FOCUS app and added sleep interventions and on-demand educational components; they found reductions in several symptom domains in schizophrenia. Finally, Bucci et al [52] developed the Actissist app, which uses an algorithm based on the cognitive model of psychosis, and found large improvements in symptoms relative to symptom monitoring only in participants with early psychosis.

Related to this, recent meta-analyses have suggested that SST may be a more effective treatment for negative symptoms of schizophrenia than CBT [28,29]. The mCBTn intervention in this study did not include the SST or problem-solving components included in CBSST. Thus, while SST may be an effective treatment for negative symptoms, this trial suggests that the CBT components of CBSST targeting defeatist attitudes can improve experiential negative symptoms without SST.

With regard to secondary outcomes, modest improvements were found in positive symptoms and depression with a longer 24-week treatment period, which was not expected, given that participants were screened for severe positive symptoms and depression. Changes in functioning were mixed, with significant improvements found early in treatment but then dissipated with only trend-level improvements found overall on the A-QLS and SFS. The 24-week follow-up period may be too brief to expect meaningful changes in functioning.

This study had a high exclusion rate during the run-in period, with 36 out of 67 (54%) participants not meeting the strict, persistent, negative symptom entry criteria. A high screen failure rate during run-in periods is common in clinical trials with similar persistent negative symptom criteria. For example, a screen failure rate of 44% was found in a psychosocial trial using similar criteria, except DPAS [26], and this rate is slightly higher than in pharmaceutical trials with similar criteria [26,53]. The proportion of patients with documented persistent negative symptoms (46%) is consistent with other estimates [3] and suggests negative symptoms are a common treatment need in patients with schizophrenia. Clinical trials of participants with persistent negative symptoms are rare, so this study makes an important contribution by demonstrating improvements in experiential negative symptoms as a primary target rather than secondary improvements related to changes in positive or depressive symptoms. It is important to note that participants were also recruited specifically for persistent experiential negative symptoms, which are more strongly linked to functional impairments than expressive symptoms [5,7,54,55].

Retention rates were excellent (81%-90% across assessments), especially for this negative symptom population, suggesting the intervention is feasible. It may be important that transportation was provided to therapy groups, which likely facilitated retention and may be necessary to maintain engagement of this population, especially in a large county with limited public transportation where this study was conducted. We have found much better retention in CBSST trials when transportation was provided [56] than when it was not [57]. The CBT2go app may have also promoted engagement, for example, through daily reminders of personalized recovery goals. The mCBTn intervention focused on recovery goal work, which can improve motivation and promote engagement in psychiatric rehabilitation [58].

Engagement with the CBT2go app was mixed. The app was designed to promote engagement in recovery activities as often as every day. On average, however, participants responded to prompts to make an action plan for the day about one and a half times per week. While this proportion of days with completed action plans may seem low, practicing skills and completing homework assignments more than once per week is greater homework adherence than would be expected with CBT group therapy alone, where participants are typically expected to complete a single homework assignment per week. Homework adherence in CBT psychosocial interventions across multiple disorders is approximately 20%-56%. Thus, completing approximately one and a half action plans per week is better community engagement in recovery activities than might be expected in CBT therapy alone with one assignment per week. Greater engagement with the app was also associated with greater improvement in motivational negative symptoms and was unrelated to baseline severity of negative symptoms, suggesting the app played an important role in strengthening the treatment's impact on this important outcome.

This trial had several limitations. First, as described above, further app development is needed to promote engagement (eg, simplified interface and rewards and feedback to motivate). In addition, patients with greater severity of defeatist attitudes were recruited, because this was the target mechanism, so the findings may not generalize to patients whose experiential negative symptoms may not be related to defeatist attitudes. Participants were also excluded for severe positive symptoms or depression, so findings may not generalize to these populations. Finally, and importantly, this was a preliminary open trial that did not control for the effects of time, therapist contact, trips out of the home to come to group, socialization with staff and other patients, and other nonspecific factors. The next step is to complete an RCT with a contact control condition, which we are currently conducting. If this ongoing RCT with the modified CBT2go app confirms the findings of this open trial, this would provide stronger support for the defeatist attitude model of experiential negative symptoms and suggest that blended interventions like mCBTn can strengthen and shorten intensive psychosocial interventions for negative symptoms in schizophrenia.

Acknowledgments

We thank the participants who volunteered for this study. Research reported in this publication was supported by the National Institute of Mental Health (principal investigator EG; R61MH110019). This open trial was funded as part of the National Institute of Mental Health Experimental Therapeutics Program (RFA-MH-18-704 R61/R33). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Department of Veterans Affairs or National Institutes of Health.

Conflicts of Interest

EG has an equity interest in Granholm Consulting, Inc, and may benefit from the research results as he receives income from the company for CBSST workshops and consulting. The terms of this arrangement have been reviewed and approved by the University of California, San Diego, in accordance with its conflict of interest policies. All other authors have no conflicts of interest to declare.

References

https://mental.jmir.org/2020/12/e24406

XSL-FO

https://mental.jmir.org/2020/12/e24406

XSL-FO

intervention (SAVVy): Feasibility, acceptability and preliminary clinical outcomes. Schizophr Res 2020 Feb;216:479-487. [doi: 10.1016/j.schres.2019.10.026] [Medline: ~31812327]

2006;30(2):129-136. [doi: 10.2975/30.2006.129.136] [Medline: 17076056]

Abbreviations

ABS: Asocial Beliefs Scale

https://mental.jmir.org/2020/12/e24406

XSL-FO

A-QLS: Abbreviated Quality of Life Scale

CAINS-MAP: Clinical Assessment Interview for Negative Symptoms Motivation and Pleasure CBSST: cognitive behavioral social skills training

CBT: cognitive behavioral therapy

CDS: Calgary Depression Scale for Schizophrenia

CONSORT: Consolidated Standards of Reporting Trials

DPAS: Defeatist Performance Attitude Scale

DSM-5: Diagnostic and Statistical Manual of Mental Disorders, fifth edition

EMA: ecological momentary assessment

HLM: hierarchical linear modeling

MATS: Mobile Assessment and Treatment of Schizophrenia

mCBTn: mobile-assisted cognitive behavioral therapy for negative symptoms

PANSS: Positive and Negative Syndrome Scale

PRIME: Personalized Real-time Intervention for Motivational Enhancement

RCT: randomized controlled trial

SAVVy: Smartphone-Assisted coping-focused interVention for Voices

SCID-5: Structured Clinical Interview for DSM-5

SFS: Social Functioning Scale

SST: social skills training

Edited by J Torous; submitted 17.09.20; peer-reviewed by M Machin, I Bell; comments to author 11.10.20; revised version received 03.11.20; accepted 03.11.20; published 01.12.20

Please cite as:

Granholm E, Holden J, DwyerK, Mikhael T, LinkP, Depp C

Mobile-Assisted Cognitive Behavioral Therapy for Negative Symptoms: Open Single-Arm Trial With Schizophrenia Patients

JMIRMentHealth 2020;7(12):e24406

URL: https://mental.jmir.org/2020/12/e24406

doi: 10.2196/24406

PMID: 33258792

©Eric Granholm, Jason Holden, Kristen Dwyer, Tanya Mikhael, Peter Link, Colin Depp. Originally published in JMIR Mental Health (http://mental.jmir.org), 01.12.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.

Journal of Behavioral and Cognitive Therapy (2020) 30, 13-21


Available online at

ScienceDirect

www.sciencedirect.com

Elsevier Masson France

EM|consulte

www.em-consulte.com/en


RESEARCH PAPER

Mobile-assisted cognitive-behavioral social skills training in older adults with schizophrenia

Check for updates


Eric Granholmab’*, Jason L. Holdena, Kristen Dwyerb, Peter Linka

a Veterans Affairs San Diego Healthcare System, 3350, La Jolla Village Drive (116B), 92161 San Diego, CA, United States

b Department of Psychiatry, University of California, 9500, Gilman Drive, La Jolla, 92037 San Diego, CA, United States

Received 19 September 2019; received in revised form 11 October 2019; accepted 27 October 2019 Available online 4 April 2020

KEYWORDS

Cognitive-behavioral therapy;

Social skills training; Serious mental illness;

Digital interventions;

Aging;

Community functioning


Abstract Psychosocial rehabilitation interventions are needed to improve functioning in people with schizophrenia, particularly for older adults. In multiple clinical trials, cognitive-behavioral social skills training (CBSST) has been found to improve community functioning in youngerand older participants with schizophrenia. To reduce the burden and possibly strengthen CBSST, we developed a mobile-assisted CBSST intervention (MA-CBSST), in which therapist contact was reduced 50% and group sessions were supplemented by a mobile device that prompted at-home practice of CBSST skills. This study was a preliminary randomized clinical trial comparing: (1) the full CBSST program; (2) MA-CBSST and (3) a device contact (DC) control (only symptom and activity monitoring) in 57 older adults with schizophrenia (age > 45). Relative to DC-only, CBSST skill knowledge and self-reported functioning improved significantly more in the full CBSST program and full CBSST and MA-CBSST groups did not differ significantly, but improvements on these outcomes were only marginally significant for MA-CBSST relative to DC-only with smaller effect sizes. The results replicated multiple prior clinical trials showing improvement in functioning in schizophrenia in CBSST, but the effect of MA-CBSST on functioning was weaker than the full CBSST program.

Published by Elsevier Masson SAS on behalf of Association Francaise de Therapie Comporte-mentale et Cognitive.

* Corresponding author at: Veterans Affairs San Diego Healthcare System, VA San Diego Healthcare System, 3350, La Jolla Village Drive (116B), 92161 San Diego, CA, United States.

E-mail addresses: egranholm@ucsd.edu (E. Granholm), jlholden@ucsd.edu (J.L. Holden), krdwyer@ucsd.edu (K. Dwyer), plink@ucsd.edu (P. Link).

https://doi.org/10.1016Zj.jbct.2020.03.006

2589-9791/Published by Elsevier Masson SAS on behalf of Association Francaise de Therapie Comportementale et Cognitive.

Introduction

Psychosocial rehabilitation interventions to improve functioning in people with schizophrenia are needed, particularly for older adults. We developed cognitive-behavioral social skills training (CBSST) to improve community functioning in younger and older people with schizophrenia (Granholm, McQuaid; & Holden, 2016). In developing CBSST for older adults, several recommendations to address the unique needs of older patients were followed (Arean, 1993; Arean, Cook, Gallagher-Thompson, Hegel, Schulberg, Schulz, 2003; Gallagher-Thompson & Thompson, 1995). We selected and combined CBT and SST techniques, because the educational, collaborative approach of these interventions can be more acceptable to older adults than other forms of psychotherapy. For older adults, we refer to sessions as "classes”, because older adults would typically prefer to attend a class focused on achieving recovery goals, than another "group”. The educational model of CBT can also allay fears of disclosure in "group therapy” common in older adults. Older adults often lose important sources of support, as friends and relatives become disabled or die, so CBSST teaches social skills to address issues of loss and improving social support and leisure activities. Repeated practice of basic procedures and neurocognitive compensatory aids are also used to help minimize treatment barriers related to neurocognitive impairment and a key component is at-home practice in the form of homework assignments.

In several clinical trials of CBSST, we found improved functioning in both younger and older adults with schizophrenia (Granholm, McQuaid, McClure, Auslander, Perivoliotis, Pedrelli, Jeste, 2005; Granholm, McQuaid, McClure, Link, Perivoliotis, Gottlieb, Jeste, 2007; Granholm, Holden, Link, McQuaid, & Jeste, 2013; Granholm, Holden, Link, & McQuaid, 2014). CBSST, however, is an intensive program of weekly sessions of up to 120 minutes for 6 to 9 months, placing significant burden on service pro-grams/financial resources, providers, and older adults for time and travel, which is particularly challenging for older adults. If the impact of CBSST on functioning could be strengthened, it may be possible to reduce in-session intervention time, which is a barrier to implementation of CBSST.

One way to strengthen the intervention may be to increase practice and use of skills in the community. Greater adherence to homework assignments in CBT and SST has been associated with improved outcomes for a variety of disorders (Cammin-Nowak, Helbig-Lang, Lang, Gloster, Fehm, &Gerlack, 2013; Coon & Gallagher-Thompson, 2002; Decker, Kiluk, Frankforter, Babuscio, Nich, & Carroll, 2016; Glaser, Kazantzis, Dean, & Oades, 2000; Ntoutsia, Katsamagkos, & Economou, 2013; Olatunji, Rosenfield, Monzani, Krebs, Heyman, Turner, Mataix-Cols, 2015). The assumption that homework is of paramount importance in CBT and SST interventions is reflected in the fact that homework assignments are an important focus of therapy manuals (Granholm, McQuaid, & Holden, 2016; Bellack, Mueser, Gingerich, Agresta, 2004; Kingdon & Turkington, 2005; Rector & Beck, 2001). We have found that patients who attempted more homework assignments in CBSST showed greater skills acquisition (Granholm, Auslander, Gottlieb, McQuaid, & McClure, 2006). Modifying interventions to promote greater homework adherence, therefore, may lead to improved treatment outcomes. To promote greater homework adherence, we developed mobile-assisted CBSST (MA-CBSST), which uses mobile devices to prompt homework practice outside of sessions. If supplementing group sessions in this way strengthens the intervention by promoting greater skills practice in the community, it may be possible reduce the intensity of CBSST (shorter or fewer sessions) to reduce time and cost burdens.

Text-messaging interventions, smartphone apps, and virtual reality have been used to strengthen and reduce the burden of intensive psychotherapy interventions, both in individual and group formats, for a variety of psychiatric problems including anxiety disorders (Kenardy, Dow, Johnston, Newman, Thomson, & Taylor, 2003; Przeworski & Newman, 2004), PTSD (Reger, Hoffman, Riggs, Rothbaum, Ruzek, Holloway, & Kuhn, 2013), depression (Burns, Begale, Duffecy, Gergle, Karr, Giangrande, & Mohr, 2011; Kramer, Simons, Hartmann, Menne-Lothmann, Viechtbauer, Peeters, Wichers, 2014), bipolar disorder (Depp, Ceglowski, Wang, Yaghouti, Mausbach, Thompson, & Granholm, 2015), and schizophrenia (Adery, Ichinose, Torregrossa, Wade, Nichols, Bekele, & Park, 2018; Ben-Zeev, Brenner, Begale, Duffecy, Mohr, & Mueser, 2014; Granholm, Ben-Zeev, Link, Bradshaw, & Holden, 2012; Schlosser et al., 2016), often resulting in shorter treatment duration or less frequent client contact (cost savings), while maintaining or improving efficacy. For example, one study found an estimated savings of $540 per person in computer-assisted individual therapy for panic (Kenardy et al., 2003), and another found $1057 per person cost savings in group therapy for generalized anxiety (Newman, Consoli, & Taylor, 1999). One large trial found improvement in daily activities for patients with memory problems stemming from neurologic disorders, with effects continuing 7-weeks after returning the device (Wilson, Emslie, Quirk, & Evans, 2001). This suggests cognitive impairment may not be a barrier to use of this technology. Given the prior success of mobile interventions and the fact that mobile device use in serious mental illness (SMI) is common (Brunette, Achtyes, Pratt, Stilwell, Opperman, Guarino, & Kay-Lambkin, 2018; Ben-Zeev, Davis, Kaiser, Krzsos, & Drake, 2013), mobile technology provides an opportunity to strengthen and expand access to evidencebased interventions for SMI.

This study is a preliminary clinical trial comparing the full CBSST program with MA-CBSST and a device contact (DC) control condition for middle-aged and older outpatients with schizophrenia or schizoaffective disorder. It was hypothesized that CBSST and MA-CBSST would both improve functioning to a greater extent than the DC-only control condition, but CBSST and DC-only would not differ significantly. We also examined whether MA-CBSST increased homework adherence and whether greater homework adherence was associated with better outcome.

Methods

Participants

Participants (n = 57) met the following inclusion/exclusion criteria:

Table 1 Baseline participant characteristics.

Variable

DC-only

(n = 14)

CBSST

(n = 26)

MA-CBSST

(n = 17)

Statistical analysis

n

%

n

%

n

%

X2       df

P

Male

13

93

18

69

16

94

5.79    2

0.055

Caucasian

8

57

15

58

10

59

0.10    2

0.995

M

SD (range)

M

SD (range)

M

SD (range)

F

df

P

Age (years)

55.5

6.2 (47—67)

55.2

7.9 (45—81)

57.6

8.0 (46—74)

0.55

2,54

0.582

Education (years)

12.2

1.8 (8—16)

12.2

2.0 (9—16)

11.5

2.6 (8—15)

0.39

2,43

0.682

PANSS Total

70.6

20.9 (42—117)

72.0

25.4 (34—148)

68.6

22.3 (35—116)    0.10

2,54

0.901

DC: device contact control; CBSST: cognitive-behavioral social skills training; MA-CBSST:

mobile-assisted CBSST; PANSS: Positive and

Negative Syndrome Scale.


Interventions

This is a randomized clinical trial comparing three treatment conditions: CBSST, MA-CBSST, and DC-only. Participants completed 6 months of intervention and were followed longitudinally for 6 months after treatment. A multidimensional evaluation of treatment outcome, including functioning (primary outcome), CBSST skills acquisition, symptoms, cognitive insight, defeatist performance beliefs, and homework adherence were conducted at baseline, mid-treatment (Week 12), end of treatment (Week 24), and 6-month followup (12 months after baseline).

Cognitive-behavioral social skills training (CBSST)

CBSST consisted of 24 weekly group sessions, each two hours in length, with a half-hour lunch break after the first hour. Groups had an average of 6—8 participants (maximum of 10). The intervention integrates CBT and SST techniques, modified for use with older adults with psychosis, as described previously (Granholm et al., 2016). The intervention includes a workbook that describes the skills and includes homework assignment forms. Cognitive therapy is combined with role-play practice of communication skills and problem-solving training. Aids to compensate for cognitive impairment common in both schizophrenia and normal aging are incorporated, such as using reminder notes, intervention workbooks with larger fonts, prompting participants to write down information during and between sessions, and acronyms to assist with recall, as well as repeated practice of skills. CBSST, therefore, targets the multidimensional deficits that contribute to disability in older adults with schizophrenia.

Mobile-assisted CBSST (MA-CBSST)

The same CBSST modules were presented in weekly 60minute group therapy sessions over 24 weeks. This is a 50% reduction in provider contact relative to the full dose of CBSST. Handheld computers (Tungsten E2 Personal Digital Assistant) were used to supplement sessions throughout the entire 24-week intervention phase, but not during follow-up. The device prompted module-specific homework adherence and participants completed very brief self-monitoring ratings (moods, voices, current activities, medication adherence) three times per day (morning, mid-day and evening). The device signalled participants with beeps and used simple text screens to ask if they wanted to do the homework from the weekly session; if they answered yes, the device suggested they use their workbook to complete the homework. If they declined, the device suggested that doing more homework would help them achieve their recovery goals. The entire digital intervention was text-based, without any graphics.

Device contact-only (DC-only)

This arm of the study was designed to provide participants with device contact and symptom-monitoring but no CBSST skills training. We have found that devices themselves can have a positive impact on participants (e.g., "I felt important like my doctor carrying this around”). We controlled for this and the effects of symptom self-monitoring by providing participants with a device that signaled them three times per day during the entire 24-week intervention phase to ask the same questions about basic symptom self-monitoring, activities and medication adherence as in MA-CBSST.

Outcome measures

Functioning

The Independent Living Skills Survey (ILSS) (Wallace, Liberman, Tauber, & Wallace, 2000) measures self-report of whether or not participants performed basic functional living skills in the past two weeks. The ILSS includes 70 items across 10 domains of functioning (rated 1 =yes, performed; 0 = no, not performed; or not able to demonstrate). We created a composite score (range = 0—1) of only 5 ILSS domains (appearance and clothing, personal hygiene, health maintenance, transportation, and leisure/community activity domains), because the other domains (e.g., cleaning, finances, employment) are less relevant for adults who are approximately retirement age and predominantly live in board-and-care and other assisted living situations, where they have no opportunity to demonstrate these skills.

The Maryland Assessment of Social Competence (MASC) (Sayers, Bellack, Wade, Bennett, & Fong, 1995) is a performance-based (behavioral role-play) measure of the ability to resolve interpersonal problems through conversation. It consists of three 3-minute role-play scenarios during which the participant interacts with a live confederate who portrays the protagonist (e.g., boss) in a problem-oriented situation. Each of the three scenarios is designed to tap a different social behavior: assertion, compromise, and conversation initiation. Three parallel sets of scenarios were used to control for multiple administrations. Participant responses were videotaped for subsequent coding by blinded raters on conversational content, non-verbal content, and overall effectiveness, with the latter used in this study.

Clinical symptoms

The Positive and Negative Syndrome Scale (PANSS) (Kay, Fiszbein, & Opler, 1987) was used to assess positive and general psychiatric symptoms. The Scale for the Assessment of Negative Symptoms (SANS) (Andreasen, 1982) was used to assess two negative symptom factors: diminished expression, defined as the sum of the affective flattening and alogia items 1, 2, 3, 4, 5, 7, 9, 11, and 12, and diminished motivation, defined as the sum of avolition-apathy and anhedonia-asociality items 14, 15, 16, 18, 19, 20, and 21.

Defeatist performance attitudes

The Defeatist Performance Attitudes Scale (DPAS) (Cane, Olinger, Gotlib, & Kuiper, 1986) is a 15-item subscale of the Dysfunctional Attitudes Scale (Weissman, 1978), which has been found to be uniquely associated with negative symptoms and functioning (Campellone, Sanchez, & Kring, 2016) in schizophrenia. Items are rated on a 1—7 Likert scale with higher total scores (range = 15—105) indicating more severe defeatist performance attitudes.

Cognitive insight

The Beck Cognitive Insight Scale (BCIS) (Beck, Baruch, Balter, Steer, & Warman, 2004) is a 15-item self-report inventory of cognitive insight using a 4-point Likert scale (0 = "Do Not Agree at All’’ to 3 = "Agree Completely’’). The BCIS yields two subscales, self-reflectiveness (e.g., "There is often more than one possible explanation for why people act the way they do’’, "I have jumped to conclusions too fast’’) and self-certainty (e.g., "My interpretations of my experiences are definitely right’’, "If something feels right, it means that it is right’’). An R-C Index score, computed as the difference between the two subscales was used.

Skills acquisition

The Comprehensive Module Test (CMT) (Granholm et al., 2016) was used to assess CBSST skills acquisition, including factual knowledge about skills (e.g., "What are the 3C’s?’’) and the application of skills in vignettes (total range = 0—33).

Homework adherence

Homework completion was rated on a 6-point Likert scale (0 = not done at all; 1 = attempted, but shows lack of clear understanding; 2 = attempted some and showed some understanding; 3 = attempted most and understood it; 4 = completed and well understood; 5 = completed and very good to excellent). The percentage of homework assignments attempted (rated > 1) and percentage attempted and understood (rated > 3) were computed.

Statistical analyses

Mixed-effects regression modelling (utilizing HLM v6.08) was used. Growth curve models predicting each level-1 outcome variable (ILSS Composite; MASC Effectiveness; SANS Diminished Motivation; SANS Diminished Expression; PANSS Positive; BCIS; DPAS; CMT) were estimated using time (measured in months centered at baseline), as a level-1 predictor and group (DC-only [reference], CBSST, MA-CBSST) as level-2 predictors of both the slope and intercept parameters. Effect sizes at the end of treatment and 12-month followup were estimated for the two treatment groups (relative to the DC-only group) by computing the HLM model-estimated mean differences for each outcome variable and dividing by the baseline assessment pooled SD for the outcome (Feingold, 2009).

Results

Sample

Ninety-six percent (n = 55) of participants were re-assessed at mid-treatment, 81% (n = 46) at end of treatment, and 60% (n = 34) at follow-up. All participants with baseline and at least one additional assessment were included in analyses, and the vast majority (n =49/57 = 86%) of participants had at least 3 of the 4 assessments. The groups did not differ significantly in dropout rates at any assessment point. Dropouts at 12 months did not differ significantly from participants with a 12-month follow-up assessment on baseline ILSS (t(53)=0.48, P = 0.634), SANS Diminished Motivation (t(54) = 1.33, P = 0.191), SANS Diminished Expression (t(55) = 1.42, P = 0.163), PANSS Positive (t(55) = 0.67, P = 0.506), BCIS (t(55) = 1.47, P = 0.148), CMT (t(55) = 1.24, P = 0.220), or DPAS (t(54) = 1.06, P = 0.293) scores. They did differ on MASC Effectiveness (t(53) = 3.71, P <0.001), with dropouts scoring higher (indicating greater social competence) on average. The final sample (n = 57) included n = 46 with schizophrenia and 11 with schizoaffective disorder. The CBSST, MA-CBSST, and DC-only treatment groups did not differ significantly with regard to any demographic characteristic (Table 1) or any outcome variable at baseline (Table 2).

Table 2 Outcome variables by group by timepoint.

Baseline

3-months

6-months

12-months

n

M (SD)

n

M (SD)

n

M (SD)

n

M (SD)

ILSS composite

MA-CBSST

16

0.70 (0.11)

15

0.71 (0.11)

13

0.73 (0.10)

8

0.68 (0.12)

CBSST

25

0.70 (0.09)

26

0.72 (0.10)

22

0.72 (0.12)

19

0.73 (0.10)

DC-only

14

0.71 (0.09)

14

0.74 (0.09)

11

0.69 (0.11)

7

0.70 (0.09)

CMT

MA-CBSST

17

4.3 (3.6)

15

7.3 (5.4)

13

8.4 (5.7)

8

4.3 (2.7)

CBSST

26

5.8 (2.9)

26

9.7 (4.8)

22

11.0 (5.3)

17

9.1 (5.8)

DC-only

14

3.9 (3.5)

14

3.1 (2.1)

11

3.3 (2.3)

6

1.7 (2.1)

PANSS positive

MA-CBSST

17

18.7 (7.8)

15

16.6 (5.5)

13

16.8 (7.6)

8

18.1 (8.7)

CBSST

26

19.0 (7.9)

26

16.8 (8.7)

22

16.8 (7.2)

19

15.7 (7.6)

DC-only

14

16.9 (6.9)

14

14.9 (7.7)

11

15.5 (5.7)

7

14.9 (6.1)

SANS Dim. motivation

MA-CBSST

17

10.6 (7.0)

15

11.7 (7.8)

13

8.8 (5.9)

8

6.9 (4.0)

CBSST

25

9.2 (7.8)

26

8.2 (6.8)

22

7.2 (6.4)

19

7.6 (6.7)

DC-only

14

11.2 (6.4)

14

6.4 (5.3)

11

8.5 (6.2)

7

9.0 (4.8)

SANS Dim. expression

MA-CBSST

17

9.5 (9.6)

15

5.6 (6.3)

13

8.5 (8.6)

8

7.0 (8.7)

CBSST

26

8.9 (9.0)

25

7.8 (7.6)

22

6.0 (6.4)

19

8.3 (7.8)

DC-only

14

12.4 (8.2)

14

8.7 (10.3)

11

12.6 (10.3)

7

5.3 (4.1)

DPAS

MA-CBSST

17

51.6 (15.7)

15

53.5 (16.0)

13

54.9 (13.9)

8

69.0 (15.6)

CBSST

26

48.1 (16.5)

26

51.9 (17.6)

22

48.5 (19.0)

19

49.9 (20.3)

DC-only

13

38.1 (13.9)

14

47.4 (14.6)

11

48.6 (13.4)

6

64.3 (25.4)

MASC effectiveness

MA-CBSST

17

3.6 (1.0)

12

3.3 (0.8)

8

3.3 (0.5)

CBSST

24

3.2 (1.0)

21

3.7 (0.9)

18

3.2 (1.1)

DC-only

14

3.6 (0.8)

11

3.3 (1.0)

6

3.4 (1.2)

BCIS Index

MA-CBSST

17

6.5 (5.7)

15

5.5 (4.9)

13

5.2 (4.9)

8

3.1 (5.0)

CBSST

26

4.0 (3.4)

25

5.0 (4.8)

22

6.4 (5.9)

19

3.1 (4.5)

DC-only

14

5.9 (4.4)

14

3.4 (4.7)

11

4.5 (5.8)

7

4.1 (4.6)

DC: device contact control; CBSST: cognitive-behavioral social skills training; MA-CBSST: mobile-assisted CBSST; ILSS: Independent Living Skills Survey; CMT: Comprehensive Module Test; PANSS: Positive and Negative Syndrome Scale; SANS: Scale for the Assessment of Negative Symptoms; DPAS: Defeatist Performance Attitudes Scale; MASC: Maryland Assessment of Social Competence; BCIS: Beck Cognitive Insight Scale.


Outcomes

Table 2 shows descriptive statistics for each outcome variable for each treatment group at each assessment point, and results from the mixed-effects regression models are presented in Table 3. Statistically significant CBSST group X time interactions were found for the primary functioning outcome (ILSS) and CMT, indicating significantly greater improvements over time for functioning (see Fig. 1) and CBSST skill knowledge in CBSST relative to DC-only. For these outcomes, effect sizes for the difference between model-estimated means of the CBSST group relative to DC-only at 12 months ranged from medium to very large (ILSS = 0.45; CMT = 2.50; Table 3). The MA-CBSST group showed a large effect size for CBSST skill learning (CMT) but did not show significant improvement in functioning relative to DC-only and the effect size for MA-CBSST (d = 0.25) was about half that for full CBSST (d = 0.45). The effect of time was also significant for SANS Diminished Expression, indicating significant improvements in these negative symptoms over time for the DC-only group. There were no significant effects for any other outcomes.

CBSST session and device adherence

The CBSST and MA-CBSST groups did not differ significantly with regard to percentage of CBSST sessions attended (CBSST: M = 0.82, SD = 0.13, range = 50-100%; MA-CBSST: M = 0.77, SD = 0.23, range = 20-100%; t(41) = 0.93, P = 0.359). The MA-CBSST and DC-only groups did not differ significantly with regard to device prompt response rates (proportion for DC: M = 0.70, SD = 0.23, range = 26-96%; MA-CBSST: M = 0.58, SD = 0.29, range = 10-100%; t(28) = 1.29, P = 0.208). Sixty percent of the total sample responded at least daily on average for 6 months of treatment (> 168 prompts; DC: 79%;

Table 3 Mixed-effects regression models with DC-only group as reference.

Outcome Measure

Variables

y

t

P

da Treat end

da

Follow-up

ILSS composite

Intercept

0.724

32.18

<0.001

CBSST

-0.018

-0.67

0.505

MA-CBSST

-0.026

-0.79

0.434

Time

-0.003

-1.49

0.141

CBSST x time

0.005

2.16

0.035

0.15

0.45

MA-CBSST x time

0.004

1.49

0.142

0

0.25

CMT

Intercept

3.82

5.66

<0.001

CBSST

3.70

4.02

<0.001

MA-CBSST

1.55

1.37

0.178

Time

-0.12

-1.11

0.272

CBSST x time

0.39

2.83

0.007

1.80

2.50

MA-CBSST x time

0.38

1.95

0.056

1.15

1.85

PANSS positive

Intercept

16.19

9.91

<0.001

CBSST

1.98

0.91

0.367

MA-CBSST

1.86

0.81

0.423

Time

-0.16

-1.52

0.135

CBSST x time

-0.05

-0.29

0.771

0.20

0.20

MA-CBSST x time

0.05

0.37

0.715

0.30

0.30

SANS Dim.

Intercept

9.28

6.15

<0.001

Motivation

CBSST

-0.41

-0.21

0.835

MA-CBSST

2.07

0.92

0.364

Time

-0.15

-0.77

0.445

CBSST x time

-0.03

-0.13

0.896

0.05

0.05

MA-CBSST x time

-0.23

-0.92

0.361

0.10

-0.10

SANS Dim.

Intercept

11.77

4.85

<0.001

Expression

CBSST

-3.51

-1.22

0.229

MA-CBSST

-3.30

-1.07

0.291

Time

-0.40

-2.52

0.015

CBSST x time

0.30

1.27

0.211

-0.20

0

MA-CBSST x time

0.26

0.86

0.396

-0.20

0.05

DPAS

Intercept

40.63

12.75

<0.001

CBSST

8.87

2.02

0.048

MA-CBSST

9.79

2.16

0.035

Time

1.27

1.77

0.082

CBSST x time

-1.33

-1.75

0.086

0.05

-0.45

MA-CBSST x time

-0.10

-0.13

0.898

0.55

0.55

MASC Effectiveness

Intercept

3.579

17.35

<0.001

CBSST

-0.278

-0.97

0.336

MA-CBSST

-0.008

-0.03

0.980

Time

-0.017

-0.52

0.607

CBSST x time

0.032

0.83

0.410

-0.10

0.10

MA-CBSST x time

0.005

0.11

0.911

0

0.05

BCIS Index

Intercept

5.02

4.87

<0.001

CBSST

-0.12

-0.10

0.920

MA-CBSST

1.31

0.85

0.399

Time

-0.11

-0.87

0.387

CBSST x time

0.06

0.41

0.686

-0.05

0.15

MA-CBSST x time

-0.13

-0.65

0.518

0.10

-0.05

CBSST: cognitive-behavioral social skills training; MA-CBSST: mobile-assisted CBSST; ILSS: Independent Living Skills Survey; CMT: Comprehensive Module Test; PANSS: Positive and Negative Syndrome Scale; SANS: Scale for the Assessment of Negative Symptoms; DPAS: Defeatist Performance Attitudes Scale; MASC: Maryland Assessment of Social Competence; BCIS: Beck Cognitive Insight Scale.

a Effect sizes were estimated for the two CBSST groups (relative to the DC-only group) by computing the HLM model-estimated mean differences for each outcome variable and dividing by the baseline assessment pooled SD for the outcome, with a positive scores indicating higher scores in the CBSST groups and negative scores indicating higher scores in DC-only.

MA-CBSST: 44%), and only four participants responded less than weekly (<24 prompts).

Contrary to predictions, however, the device in MA-CBSST did not prompt greater homework adherence. In fact, the percentage of homework assignments attempted regardless of quality was marginally significantly lower in MA-CBSST relative to the full CBSST protocol (proportion rated > 1: CBSST M = 0.60, SD = 0.27; MA-CBSST M = 0.42, SD = 0.33; t(41) = 1.97, P = 0.056, d = 0.6) and the groups did not differ in percentage of homework assignments completed with good understanding (proportion rated > 3: CBSST M = 0.29, SD = 0.22; MA-CBSST M = 0.19, SD = 0.25; t(41) = 1.44, P = 0.156, d = 0.45). In addition, collapsing across CBSST groups, correlations between percentage of homework assignments attempted (rated > 1) and change in CMT and ILSS (slope over available assessments), were not significant (r’s = -0.01 to 0.14). Finally, collapsing across CBSST groups, poorer homework adherence (rated > 1) was correlated with greater severity of SANS Diminished Motivation (r = -0.37, P<0.05) and poorer ILSS functioning (r = 0.36, P <0.05) at baseline, but was not correlated with any other outcome variable.

Discussion

This preliminary clinical trial was conducted to determine whether adding a mobile device to prompt homework practice in CBSST (MA-CBSST) with half the session training time could produce improvements in functioning that are comparable to the full CBSST protocol in older adults with schizophrenia or schizoaffective disorder. Small improvements in functioning were found in MA-CBSST, but greater improvements were found in the full CBSST protocol, and only the full protocol showed significantly greater improvement relative to DC-only. The finding of significantly greater improvement in functioning in the full CBSST program replicated two prior CBSST clinical trials with older patients with schizophrenia (Granholm et al., 2005; Granholm et al.,

Contrary to predictions, MA-CBSST did not improve homework adherence. The MA-CBSST and full CBSST groups did not differ significantly in homework adherence (regardless of quality or understanding), and the full CBSST group actually showed marginally greater quality and understanding of homework assignments. Thus, simple reminders from a mobile device to do homework did not improve homework adherence as expected. It is possible that the weaker effect of MA-CBSST on functioning was due to the lack of effect of the mobile intervention on homework adherence. Mobile interventions that do more than simply remind patients to practice skills may be more effective at improving homework adherence (e.g., using reward systems, personalized messages linking homework to goals; social networking/competition; gamification). Patients with greater severity of motivational negative symptoms also showed poorer homework adherence, so interventions that target rewards and motivation (e.g., components of PRIME) (Schlosser, Campellone, Kim, Truong, Vergani, Ward, & Vinogradov, 2016) may be more likely to be effective at improving homework adherence than simple reminders.

A number of clinical trials of mobile interventions that incorporate cognitive therapy-informed interventions, not just simple reminders or behavioral prompts, have found improvements in symptoms and functioning outcomes (Bell, Lim, Rossell, & Thomas, 2017; Alvarez-Jimenez, Alcazar-Corcoles, Gonzalez-Blanch, Bendall, McGorry, & Gleeson,

app and added sleep interventions and on-demand text and video educational components and found reductions in several symptom domains in schizophrenia. Bucci et al. (Bucci, Barrowclough, Ainsworth, Machin, Morris, Berry, & Haddock, 2018) developed the Actissist app which also uses a similar algorithm based on the cognitive model of psychosis and found large effects for negative symptoms, mood and total symptoms relative to a symptom-monitoring control in participants with early psychosis. Thus, mobile interventions that go beyond simple reminders and incorporate cognitive therapy-informed interventions may have greater impact on a variety of outcomes.

This study had a number of limitations. This was a preliminary study with a relatively small sample, and the sample was not geriatric. While the average age was over 55, the findings may not generalize to geriatric populations. Given the smaller sample, comparisons of participants with schizophrenia versus schizoaffective disorder would be underpowered. The proportion of males in each group was also unusually high, so gender comparisons were not possible and the findings may not generalize to women. An older mobile device (PDA) was used with a limited intervention that primarily provided homework reminders. The PDA was more challenging to operate, perhaps especially for older patients, than smartphones, although engagement with the device was relatively frequent (at least weekly for nearly all participants and daily for most) despite the long 24-week treatment phase. Smartphones also have greater capabilities for more complex interventions, which as noted above, may be more effective, like our CBT2go app.

Disclosure of interest

Dr. Granholm has an equity interest in Granholm Consulting, Inc., a company that may potentially benefit from the research results as he receives income from the company for CBSST workshops and consulting. The terms of this arrangement have been reviewed and approved by the University of California, San Diego in accordance with its conflict of interest policies. Clinicaltrials.gov # NCT00712075.

The other authors declare that they have no competing interest.

Acknowledgements

We thank the participants who volunteered for this study. Research reported in this publication was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Rehabilitation Research and Development Service (P.I.: Eric Granholm, Ph.D. E4876R). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Department of Veterans Affairs.

References

Adery, L. H., Ichinose, M., Torregrossa, L. J., Wade, J., Nichols,

Alvarez-Jimenez, M., Alcazar-Corcoles, M. A., Gonzalez-Blanch, C., Bendall, S., McGorry, P. D., & Gleeson, J. F. (2014). Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res, 156(1), 96-106.

Andreasen, N. C. (1982). Negative symptoms in schizophrenia. Definition and reliability. Arch Gen Psychiatry, 39(7), 784—788.

Arean, P. A. (1993). Cognitive behavioral therapy with older adults. Behav Therapist, 17, 236—239.

Arean, P. A., Cook, B. L., Gallagher-Thompson, D., Hegel, M. T., Schulberg, H. C., & Schulz, R. (2003). Guidelines for conducting geropsychotherapy research. Am J Geriatr Psychiatry, 11(1), 9—16.

Beck, A. T., Baruch, E., Balter, J. M., Steer, R. A., & Warman, D.

M. (2004). A new instrument for measuring insight: the Beck Cognitive Insight Scale. Schizophr Res, 68(2—3), 319—329.

Bell, I. H., Lim,M. H., Rossell, S. L., & Thomas, N. (2017). Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. Psychiatric Serv, 68(11), 1172—1181.

Bellack, A. S., Mueser, K. T., Gingerich, S., & Agresta, J. (2004). Social skills training for schizophrenia: a step-by-step guide (2nd ed.). New York, NY: The Guilford Press.

Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014). Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull, 40(6), 1244—1253.

Ben-Zeev, D., Davis, K. E., Kaiser, S., Krzsos, I., & Drake, R. E. (2013). Mobile technologies among people with serious mental illness: opportunities for future services. Adm Policy Ment Health, 40(4), 340—343.

Ben-Zeev, D., Brian, R. M., Jonathan, G., Razzano, L., Pashka,

N. , Carpenter-Song, E., & Scherer, E. A. (2018). Mobile health (mHealth) versus clinic-based group intervention for people with serious mental illness: a randomized controlled trial. Psychiatr Serv, 69(9), 978—985.

Brunette, M. F., Achtyes, E., Pratt, S., Stilwell, K., Opperman, M., Guarino, S., & Kay-Lambkin, F. (2019). Use of smartphones, computers and social media among people with SMI: opportunity for intervention. Commun Ment Health J, 55(6), 973—978.

Bucci, S., Barrowclough, C., Ainsworth, J., Machin, M., Morris, M., Berry, K., & Haddock, G. (2018). Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr Bull, 44(5), 1070—1080.

Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Gian-grande, E., & Mohr, D. C. (2011). Harnessing context sensing to develop mobile interventions for depression. J Med Internet Res, 13(3), e55.

Cammin-Nowak, S., Helbig-Lang, S., Lang, T., Gloster, A. T., Fehm, & Gerlack, A. L. (2013). Specificity of homework compliance effects on treatment outcome in CBT: evidence from a controlled trial on panic disorder and agoraphobia. J Clin Psychol, 69(6), 616—629.

Campellone, T. R., Sanchez, A. H., & Kring, A. M. (2016). Defeatist performance beliefs, negative symptoms, and functional outcome in schizophrenia: a meta-analytic review. Schizophr Bull, 42(6), 1343—1352.

Cane, D. B., Olinger, L. J., Gotlib, I. H., & Kuiper, N. A. (1986). Factor structure of the Dysfunctional Attitudes Scale in a student population. J Clin Psychol, 42(2), 307—309.

Coon, D. W., & Gallagher-Thompson, D. (2002). Encouraging homework completion among older adults in therapy. J Clin Psychol, 58(5), 549—563.

Decker, S. E., Kiluk, B. D., Frankforter, T., Babuscio, T., Nich, C., & Carroll, K. M. (2016). Just showing up is not enough: homework adherence and outcome in cognitive-behavioral therapy for cocaine dependence. J Consult Clin Psychol, 84(10), 907—912.

Depp, C. A., Ceglowski, J., Wang, V. C., Yaghouti, F., Mausbach,

Depp, C. A., Perivoliotis, D., Holden, J., Dorr, J., & Granholm, E. L. (2018). Single-session mobile-augmented intervention in serious mental illness: a three-arm randomized controlled trial. Schizophr Bull, 45(4), 752-762.

Feingold, A. (2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychol Methods, 14(1), 43—53.

First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1996). Structured clinical interview for DSM-IV axis I disorders. New York, NY: New York State Psychiatric Institute.

Gallagher-Thompson, D., & Thompson, K. W. (1995). Psychotherapy with older adults in theory and practice. In Comprehensive textbook of psychotherapy: theory and practice. New York: Oxford University Press.

Glaser, N., Kazantzis, N., Dean, F., & Oades, L. (2000). Critical issues in using homework assignments within cognitive-behavioral therapy for schizophrenia. J Ration Emotive Cogn Behav Ther, 18(4), 247—261.

Granholm, E., Auslander, L. A., Gottlieb, J. D., & McQuaid, J. R. S. (2006). Therapeutic factors contributing to change in cognitive-behavioral group therapy for older persons with schizophrenia. J Contemp Psychother, 36, 31—41.

Granholm, E., Ben-Zeev, D., Link, P. C., Bradshaw, K. R., & Holden, J. L. (2012). Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull, 38(3), 414—425.

Granholm, E., Holden, J., Link, P. C., & McQuaid, J. R. (2014). Randomized controlled clinical trial of cognitive behavioral social skills training for schizophrenia: improvement in functioning and experiential negative symptoms. J Consult Clin Psychol, 82(6), 1173—1185.

Granholm, E., Holden, J., Link, P. C., McQuaid, J. R., & Jeste, D. V. (2013). Randomized controlled trial of cognitive behavioral social skills training for older consumers with schizophrenia: defeatist performance attitudes and functional outcomes. Am J Geriatr Psychiatry, 21(3), 251—262.

Granholm, E., McQuaid, J. R., & Holden, J. L. (2016). Cognitive-behavioral social skills training for schizophrenia: a practical treatment guide. New York, NY: Guilford Press.

Granholm, E., McQuaid, J. R., McClure, F. S., Auslander, L. A., Perivoliotis, D., Pedrelli, P., & Jeste, D. V. (2005). A randomized controlled trial of cognitive behavioral social skills training for middle-aged and older outpatients with chronic schizophrenia. Am J Psychiatry, 162(3), 520—529.

Granholm, E., McQuaid, J. R., McClure, F. S., Link, P. C., Perivoliotis, D., Gottlieb, J. D., & Jeste, D. V. (2007). Randomized controlled trial of cognitive behavioral social skills training for older people with schizophrenia: 12-month follow-up. J Clin Psychiatry, 68(5), 730—737.

Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull, 13(2), 261—276.

Kenardy, J. A., Dow, M. G. T., Johnston, D. W., Newman, M. G., Thomson, A., & Taylor, C. B. (2003). A comparison of delivery methods of cognitive-behavioral therapy for panic disorder: an international multicenter trial. J Consult Clin Psychol, 71(6), 1068—1075.

Kingdon, D. G., & Turkington, D. (2005). Cognitive therapy of schizophrenia. New York, NY: Guilford Press.

Kramer, I., Simons, C. J., Hartmann, J. A., Menne-Lothmann, C., Viechtbauer, W., Peeters, F., & Wichers, M. (2014). A therapeutic application of the experience sampling method in the treatment of depression: A randomized controlled trial. World Psychiatry, 13(1), 68—77.

Newman, M. G., Consoli, A. J., & Taylor, C. B. (1999). A palmtop computer program for the treatment of generalized anxiety disorder. Behav Modif, 23(4), 597—619.

Ntoutsia, P., Katsamagkos, A., & Economou, M. (2013). The efficacy of social skills training for individuals with schizophrenia. Psychol J Hell Psychol Soc, 20(1), 34—53.

Olatunji, B. O., Rosenfield, D., Monzani, B., Krebs, G., Heyman,

Przeworski, A., & Newman, M. G. (2004). Palmtop computer-assisted group therapy for social phobia. J Clin Psychol, 60(2), 179—188.

Rector, N. A., & Beck, A. T. (2001). Cognitive behavioral therapy for schizophrenia: an empirical review. J NervMent Dis, 189(5), 278—287.

Reger, G. M., Hoffman, J., Riggs, D., Rothbaum, B. O., Ruzek, J., Holloway, K. M., & Kuhn, E. (2013). The "PEcoach” smartphone application: an innovative approach to improving implementation, fidelity, and homework adherence during prolonged exposure. Psychol Serv, 10(3), 342—349.

Sayers, M. D., Bellack, A. S., Wade, J. H., Bennett, M. E., & Fong, P. (1995). An empirical method for assessing social problem solving in schizophrenia. Behav Modif, 19(3), 267—289.

Schlosser, D., Campellone, T., Kim, D., Truong, B., Vergani, S., Ward, C., & Vinogradov, S. (2016). Feasibility of PRIME: a cognitive neuroscience-informed mobile app intervention to enhance motived behavior and improve quality of life in recent onset schizophrenia. JMIR Res Protoc, 5(2), e77.

Wilson, B. A., Emslie, H. C., Quirk, K., & Evans, J. J. (2001). Reducing everyday memory and planning problems by means of a paging system: a randomized control crossover study. J Neurol Neurosurg Psychiatry, 70(4), 477—482.

Wallace, C. J., Liberman, R. P., Tauber, R., & Wallace, J. (2000). The independent living skills survey: a comprehensive measure of the community functioning of severely and persistently mentally ill individuals. Schizophr Bull, 26(3), 631—658.

Weissman, A. N. (1978). Development and validation of the dysfunctional attitude scale: A preliminary investigation. Toronto, Ontario: Annual Meeting of the American Educational Research Association.

A Web-based Game for Teaching Facial Expressions to Schizophrenic Patients

Kemal Hakan Gulkesen1; Filiz Igleyen1; Buket Cinemre2; Mehmet Kemal Samur3; Semiha §en Kaya2; Nege Zayim1

Keywords

Facial expression, video games, schizophrenia

Summary

Background: Recognizing facial expressions is an important social skill. In some psychological disorders such as schizophrenia, loss of this skill may complicate the patient's daily life. Prior research has shown that information technology may help to develop facial expression recognition skills through educational software and games.

Objectives: To examine if a computer game designed for teaching facial expressions would improve facial expression recognition skills of patients with schizophrenia.

Methods: We developed a website composed of eight serious games. Thirty-two patients were given a pre-test composed of 21 facial expression photographs. Eighteen patients were in the study group while 14 were in the control group. Patients in the study group were asked to play the games on the website. After a period of one month, we performed a post-test for all patients.

Results: The median score of the correct answers was 17.5 in the control group whereas it was

Conclusions: Computer games may be used for the purpose of educating people who have difficulty in recognizing facial expressions.

Correspondence to:

Kemal Hakan Gulkesen

Department of Biostatistics and Medical Informatics Faculty of Medicine

Akdeniz University

Antalya, Turkey

Email: hgulkesen@gmail.com


Appl Clin Inform 2017; 8: 719-730

https://doi.org/10.4338/ACI-2016-10-RA-0172 received: October 16, 2016

accepted: April 27, 2017

published: July 12, 2017

Citation: Gulkesen KH, igleyen F, Cinemre B, Samur MK, §en Kaya S, Zayim N. A web-based game for teaching facial expressions to schizophrenic patients. Appl Clin Inform 2017; 8: 719-730 https://doi.org/10.4338/ACI-2016-10-RA-0172


Downloaded by: Washington University. Copyrighted material.



Facial expressions constitute an important component of non-verbal communication between people. Many psychological studies have shown that recognizing and understanding facial expressions is an important social skill [1-3]. In certain psychological disorders, such as autism and schizophrenia, the loss of these skills may complicate the patient's daily life. For instance, patients may suffer from social difficulties and communication problems [1-4]. Prior research has shown that information technology may help to develop facial expression recognition skills through educational software and “serious” games [5-7]. The primary purpose of such serious games is training or educating the users rather than entertaining them [8].

There are many facial expressions and most of them are culture-specific. A facial expression specific to one culture may mean nothing or something quite different in another cultural environment [9]. However, it is widely accepted that there are seven universal facial expressions recognized in the same manner in every culture. The feelings these facial expressions represent are anger, fear, happiness, surprise, disgust, sadness, and neutrality [10]. Internet-based educational software using these expressions may be used in any culture and may help the patient to improve facial expression recognition skills. Frommann et al. designed a study to compare 16 ‘post-acute' patients with schizophrenia and a control group. According to the study, a group of patients who were trained with software improved their performance in the identification of facial emotions [5]. Silver et al. explored the effect of emotion training exercises on the perception of facial emotional expressions. Twenty male chronic patients with schizophrenia underwent three training sessions using emotion training software, which was initially developed for autistic children and then adapted to the clinical setting. Patients were assessed before and after training with validated tests for the identification of facial emotions, differentiation of facial emotions, and working memory. They showed that brief emotion training could improve recognition of facial emotional expressions in patients with chronic schizophrenia [6]. Russel et al. investigated the effectiveness of the ‘micro-expressions training tool' (METT), which was developed to improve emotion recognition skill. Twenty patients with schizophrenia and 20 healthy control participants were involved in the study. The patients with schizophrenia showed significant improvements in emotion recognition following the training with this tool [7].

Although previous studies have shown that facial expression training software can help patients with schizophrenia, none of those were specially designed for this purpose. Disease-specific solutions can improve the outcomes, since issues in recognizing facial expressions exhibit different patterns for each psychiatric disease [11-13]. During development of a training environment for patients with schizophrenia, the general characteristics of the disease, such as impairment in social cognition, difficulties in working memory, long-term memory, attention, executive functioning, and speed of processing, should therefore be considered.

The aim of this study was to examine if a games website designed for teaching facial expressions could improve facial expression recognition skills of patients with schizophrenia.

We designed a website that includes brief facial expression education and serious games for patients with schizophrenia. The first requirement was a basic facial expressions digital photography set. Even though the basic facial expressions could be universally recognized, we preferred to prepare a digital facial expressions photography set with the participation of 40 Turkish volunteers to decrease cultural deviations. The majority of these volunteers were amateur or professional theatre players. In contrast to the general acceptance of universal face expressions, several studies indicated that recog

Downloaded by: Washington University. Copyrighted material.


nition of facial expressions could vary in different cultures [9, 14-15]. We collected photographs for each of the six basic facial expressions and one photograph with a neutral expression. A total 1001 photographs were evaluated according to three steps. At the first stage, all facial expression photographs were assessed by the research team (FI, KHG, BC, MKS and NZ) as a consequence, 561 photographs were accepted for the second stage. At the second stage, a web survey was prepared and 33 volunteers evaluated the 561 photographs. During this stage, we asked each participant to match images with the correct facial expression, and we used the images that were correctly recognized (187 photographs, consensus >97% ). We eliminated the photographs (68 images) if they were poorly recognized (consensus < 58%) by the participants. At the final stage, the remaining 306 photographs were evaluated by an additional 396 volunteers, making a total of 427 voluntary participants. For the final evaluation, the survey was announced on the Facebook and medical informatics mail groups. All participants were scored according to their consensus with the other participants. For each photograph, the user's response was noted and the number of other users with the same response was recorded. The ratio of this number and the total number of users was calculated as the consensus score for the photograph. For each user, an overall mean consensus score was calculated based upon the mean of the consensus scores for each photograph. Participants whose consensus score was lower than 0.65 were excluded from the study. Images that were recognized by the remaining participants with a high degree of consensus (>85%) were retained in the final set of photographs. The one exception was that the threshold was decreased to 75% for the fear expression, because the consensus of volunteers was relatively low in this case. The details of developing the photography set were as described in a previous study [16]. The final set included 364 photographs which were selected from a collection of 1001 photographs, taken of 40 models. The photograph set has been shared for scientific use (http://yuzifadeleri.org/expressions.htm).

ILFE was designed as an internet-based educational tool in order to make it easily accessible (http://yuzifadeleri.org/). We used Microsoft Visual Studio 2010 Professional Edition, and Microsoft Access 2010 for developing the web site. Programming language was C#, and we also used HTML, JAVA Script and Jquery libraries The ILFE includes eight serious games, which were designed using the principles of errorless learning, repetition, feature abstraction, direct positive reinforcement, and self-instruction. Additionally, all the games were designed with a consideration for some of the common characteristics of schizophrenia-like deficiency in social cognition, difficulties in working and long-term memory, attention deficit, disturbance in executive functioning, and lower speed of processing. Decisions about the flow of the games were made by a psychiatrist (author BC).

In addition, the usability of ILFE was evaluated in two stages by using heuristic evaluation and then according the protocol analysis (PA), or the “think aloud” method.

Heuristic evaluation is an evaluation of an interface by one or more experts. Evaluators measure the usability, efficiency, and effectiveness of the interface based on ten usability heuristics originally defined by Nielsen [17]. According to Nielsen, the number of the evaluators is normally three to five, since one does not gain that much additional information by using larger numbers. For our study, seven “Medical Informatics” experts and one computer and education technologies expert completed the heuristic evaluation questionnaire; the feedback from evaluators was then used to ascertain ILFE’s design problems. Experts were given a scenario (task list) before they started the evaluation. Accordingly, they identified all major and minor problems.

Although a number of usability problems were identified in the assessments of the experts, studies like the current systems must also be evaluated by real users to increase the overall system quality and usability. We used PA with real patients as soon as we had completed the heuristic evaluation.

The PA method requires participants to verbalize their thoughts, feelings, and opinions during the test. One goal of this approach is to enable the tester to get a better understanding of the participant's mental model during interaction with the interface. Previous studies have indicated that, for extensive usability evaluation methods such as PA, using a small number of subjects (for instance five) would be sufficient due to the long evaluation process [18]. Therefore, five patients with schizophrenia participated in the PA test. Prior to the assessment, the purpose of the research and PA was explained to patients and they were then given some tasks. After the PA, with the help of the data obtained from patients, we made the final alterations [19].

Downloaded by: Washington University. Copyrighted material.


Training Module

Six basic face expression images and brief explanations about these expressions were present in the training module. The module was developed for patients who wanted to practice before playing the games or between the games.

Games Module

The games designed for the ILFE have difficulty levels ranging from easy to hard (Figure 1). In order to proceed to the next screens, the patients have to answer the questions successfully. There is no playing limit for the games; they can keep trying until they find the correct answer. There are eight games, including a memory card game, and the games have been designed to be played using a mouse. None of the games requires advanced computer skills or motor abilities to do complex tasks on keyboards or other devices. The games have no time limitation except the seventh game. The psychiatrists in our development team (BC and SSK) suggested that the addition of sounds during the game might cause a distraction for the patients. Based on this suggestion, we used no sounds for the games except for applause that plays after a correct answer along with a ‘well done' message. In the case of incorrect responses, a ‘try again' message is immediately given. The patient cannot pass to the next game without giving a correct answer.

Game 1, Name this expression

The user sees an image, and they have to find the correct facial expression text that matches to the image.

Game 2, Find the correct expression

The user sees an expression text and selects the correct expression from among several images. The game consists of three levels.

Game 3, Carry the correct image

In the first level, there are four images and one expression. The user must determine which image matches the expression, and drag the image over the expression with the help of the mouse. The number of the images increases in the second level. In the third level, the number of the images and the expressions are equal.

Game 4, Match the image and the expression

The user must find the written expression and drag it to under the appropriate image. The game starts with three images and in each level it increases by one until there are 10 images. In this game, if the user makes one or more incorrect match, they fall one level, if they make all of the matches correctly, they increase one level.

Game 5, Find the same expression

In this game, there is a sample image on the left of the screen, and there are four images on the right of the screen. These images belong to different people. The user must find the same expression in the sample image from among the images on the right. The user must find the same expression from among six images on the second level and eight images on the third level.

Game 6, Find the different expression

In the first level, there are three images of the same person. Expressions in two of the images are the same. The user has to find the different expression. In the second level, the images belong to different people.

Game 7, Balloons

In this game, balloons with facial expressions slide along the screen. Each balloon disappears within a pre-specified time. The user has to find and click on the balloon which has the correct facial expression. When they complete a mission, a congratulatory message appears and gives a new mission as: “Congratulations, you have found all the HAPPY faces, now your mission is to find SAD faces.” If the user makes nine mistakes, the game starts from the beginning. The game has seven difficulty levels.

Downloaded by: Washington University. Copyrighted material.


Game 8. Memory cards game

This game needs an additional skill, memory. The user has to find the same expression in image pairs. In the beginning, all the images are reversed. The user clicks one of the boxes and sees the image then clicks another box to see the other image. If the expressions in both images match, the images stay open. If they are different, both close. Thus, the user has to remember the place of closed images to find the pairs. The number of the cards increases by level.

After completing written informed consent, 42 patients with schizophrenia took the pre-test. Four of those 42 patients recognized 20/21 or 21/21 expressions in the pre-test, thus they were excluded from this study. It is thought that the facial expression recognition skills of these patients were not affected by their disease, and no meaningful improvement would therefore be observed in these patients. Another six patients chose to leave the study; they were also excluded. The remaining sample consisted of 32 patients with schizophrenia (20 females, 12 males, mean age ± standard deviation; 37.3±9.2). All patients were diagnosed with schizophrenia according to the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Sixth Edition) [20], and all were receiving pharmacological treatment with antipsychotics at the time of the study. Patients experiencing an acute exacerbation of illness were not accepted to the study. All patients were followed up by Akdeniz University, Department of Psychiatry.

For each participant, performance of facial expression recognition was evaluated by pre-test and post-test before and after training. The patients were randomly assigned to the study group (n=18) or to the control group (n=14). There was no statistically significant difference between the study and control groups in terms of the results of the psychiatric tests, gender, age, and educational level. The assessment of emotion recognition (pre-test, post-test) was carried out through the use of a computerized test of facial emotion recognition. The tests included 21 different photographs (three for each emotion and three neutral). The participants looked at each of the photographs one by one and decided on their answer without any time restriction. Figure 2 shows a screen from the online test. Patients got one point for each correct answer. The pre-test and post-test scores of the patients were compared separately in two groups. Although the photographs of the pre-test were used in the games, the patients had never seen the post-test's photographs prior to the post-test.

The psychopathological statuses of the patients were assessed by two psychiatrists (BC and SSK) according to the Scale for Assessment of Negative Symptoms (SANS) [21-22] the Scale for Assessment of Positive Symptoms (SAPS) [23-24] and the Brief Psychiatric Rating Scale (BPRS) [25-26]. Neuropsychological assessment tools included Serial Digit Learning Test (SDLT) [27-28], the Wisconsin Card Sorting Test (WCST) [27-32], and Porteus Labirynths [33]. Our intention was to document the relation between facial expression recognition skill and the clinical features of the patients by performing these tests.

Eighteen patients participated in the training sessions during a one month period. All patients in the training group were informed by an investigator (author FI) about how to play the games. They were requested to play the games at least twice a week, on each occasion for a 60-minutes period. Nine patients (50%) had no computer or internet access in their home, so they used a computer in the hospital while they were playing the games. Logs were recorded and checked each week. If patients had forgotten to play the games, they were reminded. At the end of the one-month period, one day after the training group's last access to the system, their performance was assessed by the post-test. The control group also took a post-test one month after the pre-test. Both groups continued to take their medication with no change during the study period.

Downloaded by: Washington University. Copyrighted material.


Normality was tested with the Shapiro Wilk test. Numeric variables were compared using the Mann Whitney U test and nominal variables were compared by chi-square tests. For pair-wise analysis, the Wilcoxon test was used. Correlations were examined by the Pearson or Spearman rho tests. All of the above tests were performed using the Statistical Package for Social Sciences 19.0. All tests were two-sided, and p<0.05 was considered to be significant.

There was no significant difference between the training and control groups in terms of gender (p=0.574), education level (p=0.084), or age (p=0.413). Similarly no significant difference was observed between the groups regarding psychopathological and neuropsychological assessments. BPRS (p=0.143), SANS (p=0.764), SAPS (p=0.659), Porteus scores (p=0.593), SDLT (p=0.233) or WCST scales; WCST-Number of Trials (p=0.112), WCST-Number Correct (p=0.102)], WCST-Total Errors (p=0.983), WCST-Perseverative Responses (p=0.983), WCST-Nonperseverative Errors (p=0.722), WCST-Perseverative Errors (p=0.867), WCST-Categories (p=0.898), WCST-% Persever-ative Errors (p=0.867), WCST-Trials to Complete First Category (p=0.437), WCST-% Conceptual Level Responses (p=0.834), or WCST-Failures to Maintain Set (p=0.112).

In the pre-test, the patients most successfully recognized happy faces (96.9%) followed by nearly a similar success rate for recognizing surprised faces (95.8%). Success in recognizing angry, sad, disgusted and neutral facial expressions was 82.3%, 78.1%, 71.0%, and 64.6% respectively. The most difficult facial expression for the patients to recognize was fear (52.1%). The patients selected “surprised” instead of a correct “feared” response in 45.8% of the questions.

There were statistical correlations between pre-test scores and neuropsychological tests. Porteus (p<0.001, r=0.695), IQ (p<0.001, r=0.692), WCST-number correct (p = 0.004, r = 0.512), WCST-total Errors, p = 0.004, r = -0.512), WCST- Nonperseverative Errors (p = 0.016, r = -0.435), WCST-% Conceptual Level Responses (p=0,019, r=0,578) and WCST-Trials to Complete First Category (p=0,002, r=0,550).

In one month period, the number of sessions for each game was determined from the logs. Six games were played median two times, however the Balloons game was played 2.5 times, while the Memory Game was played 4 times by each patient.

The users were asked which game(s) did they enjoyed during the sessions, and 10 of them expressed their preferences. Seven patients stated that they mostly liked the Balloons, two of them liked the Memory Game and one liked both the Balloons and the Memory game. Additionally, seven of the patients expressed the feeling that most of the games were too easy for them.

The median pre-test score was 16.5 in the study group (minimum-maximum: 8-19, mean±stan-dard deviation: 15.6±2.8) and 17.5 in the control group (8-19, 16±3.2) on a 21-point scale (p=0.406). Median post-test scores were 20 (16-21, 19.7±1.2), and 18 (9-19, 16.5±3.1) in the study and control groups respectively (p<0.001, Figure 3). The patients' post-test and pre-test scores were compared by pair-wise analysis for each group. There was a significant difference (p<0.001) in the training group patient's scores whereas the change in the non-training group patient's scores was marginally significant (p = 0.052). The mean difference of pre and post-tests in the study group was

The benefits of serious games on health have been shown in various studies up to this point [34-36]. People who have psychiatric diseases, such as schizophrenia, Asperger syndrome or autism may have impaired recognition of facial expressions [2-5]. It has been shown by many studies that recognition levels can be increased with computer-based education software and games [6-8, 37]. However, none of these software programmes were specifically designed for patients with schizophrenia. This study describes a web-based education tool (ILFE), which was specifically developed for training patients with schizophrenia to recognize facial expressions.

Downloaded by: Washington University. Copyrighted material.


We developed a website for this aim. A new photography set of Turkish people was prepared to minimize cultural deviations. Unlike past studies [37-38]; we did not want to idealize the photographs from voluntary models. As such, we did not determine clothing, make-up, jewellery or hairdress rules for the models; rather we asked the models to come to the studio in casual style of dress. The flow of the games was designed with consideration of the characteristics of this patient group. The games were developed in a web environment to enable the patients to have easi access at any time and any place where a computer and internet access were available. As in previous studies [38-40], we observed that recognition of happy expressions was the highest compared to other facial expressions, whereas the fear was the lowest. According to Biehl, the success in recognition of happy faces may be related to its frequency in real life [39]. Indeed, it is probable that people frequently see happy expressions in their life so they can easily recognize it. On the other hand, it is known that the mesial temporal lobe structures, implicated in fear processing, are affected in schizophrenia, a possible contribution to a defect in the recognition of fear [41]. However, difficulty in the recognition of fear expression is not specific to patients with schizophrenia: it is also seen in the normal population [9, 42]. Generally, happy expressions represent the only universally recognized positive emotion, in contrast to multiple negative emotions [41]. When we separate expressions as positive and negative, our findings show that the patients with schizophrenia are more likely to recognize positive expressions than the negative ones.

We do not have specific information about schizophrenic patients' preferences for the properties of computer games in general. This study also sets out to describe the usage pattern of a game set which was designed for use in schizophrenic patients.

Peterson reports that loss of information from short-term memory begins immediately after the learning process ends. By three seconds after learning ends, 38% of the information is lost and by 18 seconds, 85% of information is lost [43]. Repetition is very important in ensuring the retention of learned knowledge. Possibly, repeating is helpful in preventing the loss of learned information. The most fundamental way of retaining such information is simple repetition: constantly repeating the information may transfer it from short-term memory to long-term memory [44]. Immediate testing after learning is the most effective way to retain knowledge [45]. With the help of these principles, we designed a series of eight games. These games were inspired by some educational games prepared for children. Six of the games were very simple: basic visual multiple choice questions or visual matches. These games were played in a very similar frequency by our patients; they were all played a median two times by the patients. The two other games, Balloons, and the Memory Game were more complex. In the Balloons game, the images were moving, and the users had to catch them in a limited time. The Memory Game needed another skill, remembering the images. The patients played these two games more, a median 2.5 times for Balloons and four times for the Memory Game. Interestingly, the patients expressed their preference for these two games. According to the results of this study, schizophrenic patients prefer more complex computer games in spite of their particular mental disadvantages. Schizophrenic patients should, therefore, be evaluated as intelligent adult individuals, albeit with certain special characteristics. Game designs for schizophrenic patients should be further investigated to add more information about this special patient group.

In spite of deficits in attention, executive function, processing speed and working and long-term memory, patients with schizophrenia are like adults without psychiatric disorders in preferring more complex computer games. Additional studies are needed to identify the optimal characteristics and complexity of games for individuals with schizophrenia.

In some studies, it is reported that a disorder of the recognition of facial expressions is positively correlated with a disorder of general cognition [46-47]. According to our findings, Porteus maze test scores, and WCST scores had a positive correlation with the recognition of facial expressions. In other words, if the patients with schizophrenia had higher visual-spatial perception scores, they were more successful in recognizing facial expressions. According to the results of this study, WCST-number correct, WCST-total Errors, WCST- Nonperseverative Errors, WCST- % Conceptual Level Responses, WCST-Trials to Complete First Category, Porteus, and IQ scores were correlated with the level of recognition of facial expressions in patients with schizophrenia. In contrast to Bryson's study [48], we did not find any relation between the perseverative errors score in WCST and recognition of facial expressions. This situation may be due to the small number of patients involved in our study; this should be investigated with more patients. However, taken together, these findings suggest a possible association of a good cognitive performance with an improved skill in recognizing facial expressions.

Downloaded by: Washington University. Copyrighted material.


After one month of training, the facial expression recognition ability of the patients was evaluated for comparison with the pre-test scores. In the study group, a statistically significant increase in the facial expression recognition score was observed. The difference between pre- and post-test scores in the control group was close to being statistically significant (p=0.052). The increase in score in the control group can be understandable, because the patients were receiving treatment. Nonetheless, the increase in the post-test scores of the study group was more prominent.

The present study was planned so as to minimize bias. The study and control groups were similar in composition, confirmed by both demographic data and psychiatric tests. Both groups were in their routine therapy protocol during the study. However, we did not plan to make another computer game for the control group to play to equalise the possible effect of playing computer games on the ability to recognise facial expressions. Playing computer games may have a positive effect on post-test scores. Half of the patients were playing the games in the hospital. Social interactions in the hospital may also have had positive effects on facial recognition ability. In future studies, researchers should plan to use another game for the control group. This is one limitation of the study. Additionally, the sample size is small, and thus results should be confirmed in future studies.

In this research, training was limited to two sessions a week, for a month. The effect of the duration of the training period may be studied in the future. Moreover, the evaluation of patient performance was conducted immediately after the end of the training period. The long term effects -and possible effects - of training on the daily life of a patient should also be investigated. Increases in facial expression recognition ability are positive, but the real aim of this training was to improve the clinical situation of the patients. A more reliable evaluation of the benefit of the games would be possible with the help of psychiatric assessment instruments. However, we do not expect a rapid improvement in patients' ability to recognize facial expressions. This skill is required for social interactions, but developing social interactions needs time. It may have a cumulative effect on the clinical situation of the patients over the course of months or years. Future studies should, therefore, evaluate patients immediately after training and at several times after training has been completed.

It is commonly known that information technology can support individual health in a variety of situations. The results of this study are promising. Computer games may be used to educate people who have difficulty recognizing facial expressions.

Multiple Choice Question

According to results of this study, what type of games do patients with schizophrenia like?

Correct answer: A)

Explanation: Six of the games were very simple; basically just visual multiple choice questions or visual matches. These games were played at a very similar frequency by our patients (median two times). Two other games, Balloons, and the Memory Game were more complex. In the Balloons game, the images were moving, and the users had to catch them in a limited time. The Memory Game needed another skill: remembering the images. The patients played these two games more often, a median 2.5 times for the Balloons and four times for the Memory Game. According to the results of this study, schizophrenic patients prefer more complex computer games.

Downloaded by: Washington University. Copyrighted material.


Clinical Relevance Statement

The use of computer games, or in other words serious games, should be investigated to improve individual health in those with various medical conditions.

Conflict of Interest

The authors of this study report no conflicts of interest.

Human Subjects Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and the study protocol was approved by Akdeniz University Ethical Committee of Clinical Research.

Downloaded by: Washington University. Copyrighted material.





Downloaded by: Washington University. Copyrighted material.


References

25.Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychol Rep 1962; 10: 799-812.

26.Soykan Q. Institutional differences and case typically related to diagnosis, symptom severity, prognosis and treatment. MSc, Middle East Technical University, Ankara, Turkey, 1990.

36.Santamaria JJ, Soto A, Fernandez-Aranda F, Krug I, Forcano L, Gunnard K, Kalapanidas E, Lam T, Raguin T, Davarakis C, Menchon JM, Jimenez-Murcia S. Serious games as additional psychological support: A review of the literature. J Cyber Ther Rehabil 2011; 4: 469-476.

37.Silver M, Oakes P. Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism 2001; 5: 299-316.

46.Sachs G, Steger-Wuchse D, Kryspin-Exner I, Gur RC, Katschnig H. Facial recognition deficits and cognition in schizophrenia. Schizophr Res 2004; 68: 27-35.

Psychiatry Research 284 (2020) 112695



Contents lists available at ScienceDirect

Psychiatry Research

journal homepage: www.elsevier.com/locate/psychres

An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders

Ecthnr LJo n cc on a,* Qonno Roluorfa Mornroof                  ^Arlzolmcinc^      i/on

Esuiei nanssen , Sanne Balveit , iviaigieei uoisciioi , Kaiel Buikclmans , Jim van us ,

Philippe Delespaulc,e, Anne-Kathrin Fetta,f,g

a Department of Clinical, Neuro and Developmental Psychology, Faculty of Behavioural and Movement Sciences, and Institute for Brain and Behaviour Amsterdam, Vrije Universiteit Amsterdam, the Netherlands b GGZ Delfland, Delft, The Netherlands

c Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University Medical Centre, Maastricht, The Netherlands

d Department of Psychiatry, Brain Centre Rudolf Magnus, University Medical Centre Utrecht, Utrecht, The Netherlands

e Mondriaan Mental Health Trust, Heerlen, The Netherlands

f Department of Psychology, City University of London, London, United Kingdom

g CSI Lab, Institute of Psychiatry, Psychology and Neuroscience, Department of Psychosis Studies, King's College London, London, United Kingdom

ARTICLE INFO

ABSTRACT


Keywords:

Psychoses

Experience sampling method

Mobile health

Treatment

Intervention

Social contact


This study examined the feasibility and effectiveness of an interactive smartphone application that aimed to improve daily-life social functioning and symptoms in schizophrenia spectrum disorders (SZ) with Experience Sampling Method (ESM) derived personalised feedback.Two groups of outpatients with a diagnosis of SZ were included (one receiving ESM-derived personalised feedback (n = 27) and one without feedback (n = 23)) and used the interactive smartphone application for three weeks. Main outcomes were momentary symptoms and social functioning, as assessed by ESM questionnaires. Additionally, feasibility and user-friendliness of the application were assessed. The response rate was 64% for the ESM questionnaires. In the feedback group, participants indicated that on 49% of the ESM days they acted on at least one personalised feedback prompt per day. Momentary psychotic symptoms significantly decreased over time only in the feedback group. Momentary loneliness and questionnaire-assessed psychotic symptoms decreased over time, irrespective of feedback. Participants evaluated the app as user-friendly and understandable. Momentary personalised feedback may impact momentary psychosis in daily life. Feelings of loneliness and questionnaire-based measured psychotic symptoms may be more responsive to non-specific effects of daily-life self-monitoring, not requiring specific feedback. Ecological momentary interventions offer opportunities for accessible and effective interventions in SZ.

1. Introduction

Schizophrenia spectrum disorders (SZ) are characterised by social and community dysfunction (Couture et al., 2006; Garrido et al., 2013). In addition to positive and negative symptoms like hallucinations, delusions and anhedonia, difficulty in navigating the social world has a substantial impact on daily-life functioning (Couture et al., 2006Fett et al., 2011; Velthorst et al., 2016). This is reflected in key characteristics of the disorder, e.g. social withdrawal and poor social interactions (Billeke and Aboitiz, 2013; Penn et al., 1996), as well as

difficulties in maintaining relationships with family and friends (Burns and Patrick, 2007; Pinkham and Penn, 2006). Functional and social impairments remain a challenge to treat (Robinson et al., 2004Wykes et al., 2008). If social functions are targeted in interventions, effects often do not transfer to daily life (Couture et al., 2006; Pos et al., 2019; Roberts and Velligan, 2012), which may be related to the low (social) motivation associated with a diagnosis in the schizophrenia spectrum (Medalia and Saperstein, 2011). Supplementing treatment with support in real life may lead to greater functional improvement (Berry and Haddock, 2008; Bradshaw et al., 2007). This is, for instance, implemented by the Social Cognition and Interaction Training (SCIT) (Penn et al., 2007), which facilitates practice outside the therapy sessions. The SCIT shows promising results on social functioning. An easy, useful, and less resource intensive way to improve social functioning in the context of daily life for patients with a SZ diagnosis may lie in further integration with mobile health applications.

* Corresponding author: Vrije Universiteit Amsterdam, Faculty of Behavioural and Movement Sciences, Van der Boechorststraat 7, 1081 BT Amsterdam The Netherlands.

E-mail address: esther.hanssen@vu.nl (E. Hanssen).

https://doi.org/10.1016Zj.psychres.2019.112695

Received 29 July 2019; Received in revised form 10 November 2019; Accepted 16 November 2019

Available online 28 November 2019

0165-1781/ © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.Org/licenses/BY/4.0/).


Mobile phone ownership and the willingness to engage with mobile health (mhealth) is growing in populations diagnosed with a mental health disorder and up to 81% of patients diagnosed with schizophrenia own a smartphone (Firth et al., 2015; Lim and Penn, 2018; Visser et al., 2018). One of the most widely-used and validated methods to monitor experiences and behaviour in the flow of daily life is the Experience Sampling Method (ESM), also called Ecological Momentary Assessment (Delespaul, 1995; Granholm et al., 2011, 2007; Myin-Germeys et al., 2009). In ESM, participants answer a set of questions several times a day at random intervals, which allows for real-time monitoring of behaviour, mood, symptoms and context. Incorporating ESM in mhealth interventions provided promising opportunities in promoting health behaviour in the general population (Heron and Smyth, 2010) and more recently, in psychiatric disorders (Granholm et al., 2007Hartmann et al., 2015; Kramer et al., 2014; Myin-Germeys et al., 2016Visser et al., 2018). For example, prodromal symptoms of relapse in schizophrenia were identified successfully through monitoring of fluctuations in momentary symptoms, causing a reduction of the number of hospitalizations by 60% (Spaniel et al., 2008). Another mobile intervention study offered pre-scheduled and tailored interventions targeting voices, mood, sleep, social functioning and medication use. After using the application for one month patients showed a decrease in psychotic symptoms, depression and general psychopathology (Ben-Zeev et al., 2014). Others showed that sending automated pre-programmed personalised text messages in response to ESM entries increased social interactions (Granholm et al., 2011). In addition, motivational aspects in daily life were succesfully targeted through a mobile intervention in an early psychosis sample; improving self-reported symptoms of depression, defeatist beliefs, self-efficacy, and showing a marginal increase in motivation and pleasure (Schlosser et al., 2018). While these studies yielded initial evidence of beneficial effects, they did not include an ESM control group. Research shows that patients often experience a therapeutic effect through monitoring of their experiences and behaviour, whether they use mobile devices or a paper and pencil method. Monitoring symptoms in daily life during cognitive behavioural therapy improves the outcome of the treatment (Firth and Torous, 2015; Os et al., 2013; Torous and Firth, 2016). One study investigated a single-session intervention augmented by automated prompts on a mobile device in serious mental illness (schizophrenia and bipolar disorder). Three groups were included: with and without personalised cognitive behavioural therapy (CBT) prompts and a treatment as usual (TAU) group. The intervention resulted in modest, yet sustained improvement in general psychopathology, measured by questionnaires, in both CBT groups; with and without automated prompts. Incorporating personalised elements of CBT through automated prompts had an additional positive impact on community functioning and defeatist attitudes (Depp et al., 2018).

To disentangle symptom monitoring effects and personalised feedback effects, the current randomized controlled study included an experimental group that received personalised feedback prompts in response to their answers on the ESM questionnaires and an ESM control group that did not receive such feedback. We were interested in finding out whether using an interactive smartphone application is feasible in a SZ sample and whether providing personalised ESM-derived feedback can ameliorate symptoms and improve social functioning. We tested the corresponding hypotheses: (1) the application would be usable and understandable, and (2) the interactive feedback group compared to the no-feedback group would show larger improvements over time in momentary symptoms and social functioning, as measured by ESM, and symptoms and social functioning, as measured pre- and postintervention by questionnaire-based measures.

2. Methods

Sixty-four individuals with a SZ diagnosis were included. Participants were recruited through: (1) research collaborators; (2) assertive community treatment teams, i.e. GGZ inGeest, GGZ Delfland, Mentrum, Arkin, Altrecht, Dijk en Duin, and Yulius; (3) hospitals, i.e. Amsterdam Medical Centre, University Medical Centre Utrecht; and (4) with the help of patient- and relative associations, i.e. Anoiksis, Ypsilon, Phrenos, and PsychoseNet. Inclusion criteria for all participants were:

Of the 64 participants enrolled, 14 dropped out of the study for a variety of reasons: one participant was excluded because of the wrong diagnosis, and for four participants data was lost due to technical errors during the automated data transfer. There were nine non-completers, three of whom withdrew from the study due to various (personal) reasons, and six of whom completed fewer than 30% of the ESM questionnaires. Some additional information on the subjective experiences of these latter six participants are summarized in supplement A. There were no significant differences between completers and noncompleters on any of the investigated demographic or clinical characteristics (see supplement B - Table 1). The final data analysis of this study included 50 participants.

The Positive and Negative Syndrome Scale (PANSS; Kay et al. (1987)) was used to assess positive and negative symptoms in the two weeks before testing to get a baseline measure of symptoms. Participants’ subclinical self-reported positive and negative psychotic symptoms one week prior to testing were assessed with the Community Assessment of Psychic Experiences (CAPE; (Konings et al., 2006); Stefanis et al. (2002)). This self-report measure is more sensitive to pick up on subtle changes during the three-week intervention period. The Social Functioning Scale (SFS; Birchwood et al. (1990)) was included to assess social functioning in the domains social withdrawal, interpersonal functioning, recreation activities and pro-social activities. Two subtests of the Wechsler Adult Intelligence Scale (WAIS-III; Wechsler et al. (1997)) were used as an indicator of general cognitive ability: the vocabulary subtest, a verbal comprehension task, and the letter- and number span subtest, a working memory task.

The SMARTapp (Schizophrenia Mobile Assessment and RealTime feedback application) was made using custom questionnaires which were built on the PsyMate™ platform (www.psymate.eu), which is a platform including a smartphone app, a cloud-based data storage and a reporting module, that allows customized collection of ESM data (thoughts, feelings, and behaviour) in everyday life. Research has shown that patients found the PsyMate™ application user-friendly and that it is easily accessible even for people who are not acquainted with smartphones and its applications (Myin-Germeys et al., 2011). Participants were randomly assigned to either of two groups: (1) one where the SMARTapp provided feedback according to the participants’ daily ESM entries, or (2) one where the SMARTapp included only ESM questionnaires without personalised feedback.

Table 1

Demographics and baseline clinical characteristics.

Feedback n =27

Mean (SD)

No-Feedback

n = 23 Mean (SD)

Statistic

P

95% CI

Age

37.9 (8.6)

40.3 (10.9)

b =

-.007

.38

[-.02, .008]

Gender, male (n, %)

18 (66.7)

14 (60.9)

x2=

0.18

.67

Living status

x2=

4.85

.18

Alone n (%)

21 (77.8)

15 (65.2)

With partner and or children n (%)

3 (14.3)

4 (17.4)

With family/friends/roommate n (%)

3 (11.1)

1 (4.35)

Other n (%)

-

3 (13.0)

Working status %

x2=

1.95

.58

Employed n (%)

7 (25.9)

7 (30.4)

Unemployed n (%)

4 (14.8)

4 (17.4)

Unstructured activities n (%)

10 (37.0)

8 (34.8)

Other n (%)

6 (22.2)

4 (17.4)

WAIS Vocabulary

45.3 (10.8)

45.6 (10.8)

b=

-.0007

.92

[-.01, .01]

WAIS Letter number span

10.3 (2.1)

9.7 (3.0)

b=

.02

.44

[-.04, .08]

Diagnoses %

x2 =

1.99

.58

Schizophrenia n (%)

12 (44.4)

14 (60.9)

Schizoaffective disorder n (%)

10 (37.0)

6 (26.1)

Psychotic disorder n (%)

4 (14.8)

3 (13.0)

Schizophreniform disorder n (%)

1 (3.7)

-

Medication %

x2 =

4.51

.11

Atypical antipsychotics n (%)

18 (66.7)

21 (91.3)

Typical antipsychotics n (%)

6 (22.2)

1 (4.4)

None n (%)

3 (11.1)

1 (4.4)

PANSS

general [range 16-112]

28.1 (7.6)

28.7 (6.4)

b=

-.003

.77

[-.02, .02]

negative [range 7-49]

12.0 (4.3)

15.5 (5.6)

b=

-.03

.02

[-.06, -.006]

positive [range 7-49]

15.2 (5.5)

15.1 (6.1)

b=

-.0007

.96

[-.02, .02]


All participants completed up to six short ESM questionnaires daily when prompted by a beep, for a duration of three weeks. In the morning, all participants received a medication and morning hygiene reminder. The ESM-beeps occurred semi-randomly between 10:00 and 22:00; within time blocks of 130 min to ensure accurate representation of the flow of their daily lives. Symptoms, social activities and mood were assessed. Furthermore, participants were asked to fill in one additional evening questionnaire before they went to bed (available from 20:00 until 04:00). This questionnaire asked general questions about their day (e.g., “I have been alone for most part of the day”), and whether using the application had influenced their day. Questions were answered on a 7-point Likert scale, by fixed answer choices or with a binary yes/no answer. Items that were used to measure social functioning included questions about social engagement, feelings of exclusion and loneliness (see supplement C - Table 2 for ESM questions). Symptoms were assessed in the domain of psychotic experiences and positive and negative affect (e.g. cheerful, relaxed, irritated, ruminating) as previously used by others (Kramer et al., 2014; Myin-Germeys et al., 2009).

2.4. Personalised feedback vs no-feedback group

The SMARTapp was identical for both groups, except that one group received personalised interactive ESM-derived feedback from the application in the form of two tailored prompts a day. The prompts provided suggestions for a certain activity or behaviour change, depending on the previous ESM answers. The application provided feedback in the following categories: (a) psychotic symptoms, (b) social engagement,

All participants received written information by mail or e-mail prior to the first visit. They were asked to complete a personal-items-checklist regarding their favourite activities, coping mechanisms and social contacts, and to bring this list with them to the first (baseline) session. Testing took place at the VU University Amsterdam. Participants first gave written informed consent and then completed a battery of clinical measures (see Fig. 1). Participants who did not own a smartphone (14% across completers and non-completers), were provided with one (model: LG K120E), and for them additional training was provided on how to use the smartphone. The application was personalised for all participants, both with and without feedback, according to the personal preferences of the participant. For instance, participants filled in enjoyable activities, several social contacts, comforting thoughts and relaxing activities. They could access the comforting thoughts and relaxing activities at any time in the application. Other information, i.e. enjoyable activities and social contacts, were used to provide personalised feedback (see supplement D for personal list and coping tips). After this, the different elements in the SMARTapp were explained, as well as the meaning of the questions and response options and participants completed a practice ESM questionnaire together with a researcher. Participants were instructed to carry their phone with them and to complete the ESM questions whenever possible. They received written information about the study to take home.

Participants used the application for a period of 21 days. On day two and day seven participants were contacted by phone to check for technical difficulties and whether they had any additional questions. A contact number was provided for technical support. All data was automatically uploaded to a secure server according to the EU data protection guidelines.

After three weeks participants attended the second session during which they completed the post-measures (see Fig. 1). To make sure that the load of the first session was not too much, we assessed the WAIS in the second session. Participants were then asked about their experiences with the application to assess feasibility and after this they were


debriefed about the two conditions and the purpose of the study. After revealing their SMARTapp version, participants in the no-feedback group were offered to continue using the application with interactive feedback. All participants were given 150 Euro for study participation.

2.6. Data analysis

Statistical analyses were conducted using STATA 14.1 (StataCorp, 2015). To inspect the differences between groups on demographics and clinical characteristics at baseline regression analyses and chi-square tests were used.

For the ESM questions, a mean per beep was calculated for each participant for psychotic symptoms (‘suspicious’, ‘disliked’, ‘harmed’, ‘voices’, ‘apparitions’), positive affect (‘cheerful’, ‘relaxed’, ‘content’), and negative affect (‘irritated’, ‘sad’, and ‘ruminating’). These were used as dependent variables, as were social functioning outcomes (‘prefer not to be alone’, ‘feeling excluded’ and ‘feeling lonely’), the evening question (‘I have been alone for the most part of the day’) and questionnaire outcomes (CAPE and SFS). Mixed multilevel regression analyses were used to account for repeated observations within subjects (minimum of 38 per participant, 30% of the beeps in 21 days), with group (feedback vs. no-feedback) and time (all ESM questionnaires over time/baseline -post intervention) and their interaction as independent variables. In a similar fashion, logistic multilevel regression analyses were run to examine being alone (yes/no) over time.

3. Results

3.1. Demographics and baseline symptoms

Participant demographic information and clinical characteristics are shown in Table 1. The feedback and the no-feedback group differed in baseline negative symptoms; the feedback group had a lower negative PANSS scale score than the no-feedback group. The CAPE and SFS baseline scores are displayed in Table 2. The feedback group had a significantly higher score on interpersonal functioning at baseline measured with the SFS (b = -0.71, 95%CI [.16, 1.27], p = .01), all other CAPE and SFS baseline scores did not differ between groups (all p > .21).

3.2. SMARTapp use

The completers replied to 80 beeps (SD = 22.3) over three weeks (64%). The minimum was 40, maximum 126 (of 126). Including the six non-completers, who were dropped because of too little beeps, an average of 74 beeps (SD = 27.1) were completed (59%). No significant differences were found for completion between the feedback and nofeedback group (p = .76). The same pattern was found including the six non-completers (p = .81). The completion rate for evening questionnaires was 18 (SD = 4.1 or 84% (range 2 to 21). There were no significant differences between the feedback and no-feedback group (p = .40).

At the end of the day, the interactive version of the application asked participants whether they acted on the feedback suggestions. Participants reported that on 49% of the ESM days they followed at least one of the two suggestions they got from the personalised prompts. The percentage of given feedback in each category was: (1) psychotic symptoms 7.5%, (2) social engagement 17.1%, (3) health behaviour 10.9%, (4) recreational or physical activity 44.1%, and (5) mood 20.3%.

3.3. Change in momentary symptoms and social functioning

Averages of ESM outcomes per week are displayed in Table 3. There was a significant group-by-time interaction for momentary psychotic symptoms measured by ESM (b = -0.005, 95%CI [-0.01, -0.0006], p = .03). Analysis by group showed that psychotic symptoms significantly decreased in the feedback group (b = -0.003, 95%CI [-0.006, .-0005], p = .02), Cohen's d = -0.30 (week 1 to week 3). This decrease was not found in the no-feedback group (b = 0.002, p = .31). No group-by-time interaction or main effects in the model without the interaction were found for positive or negative affect (all p > .24).

For the preference not to be alone or feeling excluded by others there was no group-by-time interaction or main effect of group or time in the model without the interaction (all p > .34), nor was there any effect on being alone measured by the evening questionnaire (all p > .10). There was no group-by-time interaction and, in the model without the interaction, no group effect on loneliness (both p > .48), however, loneliness did decrease significantly over time in both groups

Table 2

Pre- and post-intervention levels of questionnaire-based measures of symptoms and social functioning and their change over time.

Feedback (n = 27)

Mean (SD)

Pre-intervention

Mean (SD)

Post-intervention

No-Feedback (n = 23)

Pre-intervention

Post-intervention

CAPE

positive [range 20-80]

29.3 (9.5)

27.2 (8.8)

1*

30.9 (9.9)

27.8 (7.0)

1*

negative [range 14-56]

27.5 (8.6)

25.3 (8.0)

27.6 (8.8)

26.9 (8.9)

depressive [range 8-32]

SFS

14.9 (4.8)

13.8 (3.8)

15.7 (6.2)

15.2 (5.8)

social withdrawal [max. 15]

9.9 (2.5)

10.1 (2.4)

9.3 (2.9)

9.3 (3.1)

interpersonal functioning [max. 9]

6.9 (0.9)

6.9 (0.9)

6.2 (1.1)

6.1 (1.9)

prosocial activities [max. 66]

17.9 (9.4)

17.4 (9.8)

14.5 (9.2)

13.6 (7.6)

recreational activities [max. 45]

21.0 (5.3)

20.8 (4.9)

20.4 (7.7)

19.3 (7.2)

* significance level p < .05

Note. The arrows point to the direction of the effect.


(b = -0.004, 95%CI [-0.007, -0.0009], p = .01), Cohen's d = -0.11 (week 1 to week 3). Multilevel logistic regression analyses showed no significant group-by-time interaction, or main effects on being alone in the model without the interaction (all p > .69).

3.4. Change in questionnaire-based measures of symptoms and social functioning

We examined the effect of group on questionnaire measures for symptoms and social functioning (for pre- and postscores see Table 2). For CAPE positive symptoms there was no group-by-time interaction or main effect of group in the model without the interaction (both p > .59), however, there was a main effect of time (b = -2.5, 95%CI [.20, .32], p < .01), showing less positive symptoms post-intervention in both groups. For the negative and depressive dimension there were no significant interaction or main effects (all p > .08).

There was no group-by-time interaction for SFS interpersonal functioning (p = .81), however, in the model without the interaction, there was a significant effect of group on the SFS interpersonal functioning subscale (b = .76, 95%CI [.20, .32], p < .01), indicating that the feedback group had higher baseline and post-intervention levels of interpersonal functioning, which did not change over time (p = .83). The SFS subscales social withdrawal, prosocial activities or recreational activities did not show any significant change (all p > .13).

3.5. Participant evaluation of the application

Participants rated the SMARTapp as easy to use (94%) and appealing (95%), indicated that questions were clear (80%), and generally felt that they could reflect their experiences well through the questions provided by the application (68%). Seventy-four percent of the participants said they used the coping tips, and 54% found them useful (43% neutral, 3% not useful). In the no-feedback group, 38% found the application annoying at some point compared to 73% in the feedback group (significantly different, x2 = 5.06, p = .03), for example, some participants indicated that there were too many beeps during the day and that they sometimes felt disturbed in their activities by the beep.

4. Discussion

This ecological momentary intervention study aimed to investigate whether usage of an interactive smartphone application with personalised feedback was feasible in SZ and whether it would improve psychotic symptoms and social functioning. One group received personalised ESM-derived feedback, while the other group received the ESM questionnaires without any personalised feedback, to disentangle the ESM and feedback effects. The findings indicate good feasibility, with high compliance to the application that was rated as user-friendly and understandable. Receiving personalised feedback was associated with a reduction in momentary psychotic symptoms, measured in daily life, in comparison to the no-feedback group. Regardless of whether participants received feedback or not; feelings of loneliness decreased and psychotic symptoms as measured by the CAPE questionnaire decreased.

4.1. Effect of the SMARTapp on symptoms and affect

As hypothesized, momentary psychotic symptoms showed a significant decrease over time in the feedback, but not in the no-feedback group, suggesting a beneficial effect of the provided prompts. While the no-feedback group showed no changes in momentary psychotic symptoms, a positive effect on psychotic symptoms in both groups was found on the CAPE questionnaire, showing that psychotic symptoms declined

Table 3

ESM outcomes by week for the no-feedback and feedback group.

ESM outcomes


Week 3


Week 3


Feedback (n = 27)

Week 1           Week 2

Mean (SD)

No-feedback (n = 23)

Week 1           Week 2

Mean (SD)

Psychotic symptoms

1.48 (0.86)

1.34 (0.66)

1.26 (0.59)

1*

1.62 (0.79)

1.62 (0.89)

1.72 (0.96)

Positive affect

5.19 (1.11)

5.21 (1.16)

5.31 (1.18)

4.81 (1.51)

4.96 (1.58)

4.81 (1.60)

Negative affect

2.01 (1.13)

2.01 (1.20)

1.91 (1.07)

2.28 (1.38)

2.14 (1.38)

2.28 (1.43)

Loneliness

2.15 (1.48)

2.08 (1.49)

1.91 (1.33)

1*

2.53 (1.78)

2.26 (1.69)

2.44 (1.74)

Feeling excluded

1.78 (1.26)

1.74 (1.25)

1.62 (1.17)

1.88 (1.37)

1.98 (1.48)

1.92 (1.37)

Prefer not to be alone

2.83 (1.65)

2.98 (1.72)

2.70 (1.74)

3.06 (1.94)

3.09 (2.07)

3.26 (2.16)

Being alone

59.9%

62.4%

62.1%

55.3%

56.4%

55.2%

Evening questionnaire

Alone most of the day

3.16 (1.89)

3.04 (1.85)

2.84 (1.79)

2.93 (1.96)

2.99 (1.89)

2.92 (1.82)

* significance level p < .05

Note. The arrows point to the direction of the effect.


1*


after three weeks. It may be that the no-feedback group, in retrospect, subjectively rated positive symptoms as being lower in the last 3 weeks, while this was not confirmed by the daily ESM entries, possibly reflecting differences between in the moment and retrospective ratings (Moran et al., 2017). The difference may be related to the reliance on patients' long-term memory about their experiences or feelings in the previous weeks. Prospective measurements better reflect the actual mental states. Accumulated sampled measurements best reflect the mental state during the period. Contrary to our hypothesis, no significant group difference was found for negative symptoms measured by the CAPE. Both groups showed a decline in negative symptoms, although this did not reach significance (p = .075).

Momentary positive or negative affect did not change over time and did not differ between groups. This may be related to relatively high average of positive affect and a low average of negative affect at the beginning of this study (e.g. ceiling and floor effects) (Huppert et al., 2001) or it may be that the application does not impact on affect, which seems to be in line with results from an ecological momentary intervention study in depression (Hartmann et al., 2015).

4.2. Effect of the SMARTapp on social functioning

We found a decrease in loneliness over time in both groups, as indicated by ESM entries. We did not find an effect on social engagement (i.e. being alone). This does not support the hypothesis that participants in the feedback group would show greater improvement in social engagement than the no-feedback group. However, the found decrease in loneliness is important, because loneliness ratings amongst individuals with a schizophrenia spectrum disorder are high (up to 80%) (Stain et al., 2012) and loneliness is a significant contributor to quality of life and subjective well-being (Eglit et al., 2018). The decrease in loneliness may be partly explained through use of the application itself, related to the monitoring of experiences or coping tips (Firth and Torous, 2015; Os et al., 2013; Torous and Firth, 2016) or by the regular contact with the research team. During the evaluation of the SMARTapp, some participants indicated that bacause of the application ‘it felt like someone was there for them' and ‘someone listened to them' while using the application. In addition, participants may be more inclined to enrol in a treatment study when they are more symptomatic and therefore, these improvements in loneliness, and in positive symptoms measured by the CAPE, could possibly reflect a relative turn towards the better during the fluctuating course of their illness. Future research with a waitlist control group will be necessary to disentangle these effects.

Overall and distinct domains of social functioning, as measured by the SFS, did not change over time and did not show a differential effect of feedback vs. no-feedback. Other research also failed to find effects of interactive feedback on questionnaire-based assessments of symptoms and functioning (Granholm et al., 2011), but did find an effect on dailylife social engagement in a 12-week intervention. It is possible that questionnaire measures may not be sensitive enough to detect subtle changes in social functioning or that the study period of three weeks was too short to have a significant beneficial effect on social interactions. Integrating more sensitive measures, e.g. performance-based measures of social competence, might be more successful in detecting changes in functioning (Bowie et al., 2008) and it may be helpful to include (social) motivational aspects specifically in a mobile intervention to increase social engagement (Schlosser et al., 2018). In addition, integrating mobile sensing, i.e. acquiring data from the environment through a smartphone, may be useful to detect subtle changes in activity levels in an objective way, through geolocations or telephone calls in patients' daily live context (Ben-Zeev et al., 2015; Seppala et al., 2019). Future studies including personalised feedback may benefit from incorporating video's as feedback, since studies show that patients prefer video interventions because they are experienced as more personal, engaging, and helpful than written interventions (Ben-Zeev et al., 2018).

4.3. Feasibility of the SMARTapp

The mobile phone ownership of participants in this study was high (86%) and in line with previous literature (Firth et al., 2015; Lim and Penn, 2018; Visser et al., 2018). Results on the feasibility of the application were generally positive and compliance was high (64% of the ESM questionnaires and 84% of the self-initiated evening questionnaires). The completion rates did not change over the three week course (63.5%, 61.3% end 66.7% respectively). Also, patients receiving ESM-derived feedback attempted to apply suggestions to their daily lives. Participants generally found the application easy to use, appealing and the questions clear and easy to understand. The feedback group indicated more annoyance from the application, which may be related to a higher number of beeps in total compared to the no-feedback group causing more irritation and disruptions in daily life. Not all participants indicated a reason for feeling annoyed; therefore we cannot pinpoint the precise causes. However, some participants indicated that they sometimes received feedback that was not relevant at the time that they received it. For example, receiving feedback about contacting someone after being alone for most part of the day may not be relevant anymore if the participant just visited a friend or family member. On the other hand, we speculate that participants' annoyance may increase because they find it hard to find the motivation to call someone or to go and be active, even after receiving a feedback suggestion. Future studies should elucidate what the optimal number of beeps is to foster continuous engagement with the app, but not disturbance. In addition, feedback options may be enriched by advice from the patient community, to ensure more relevant and creative suggestions.

4.4. Limitations and future directions

Some limitations must be considered with respect to the study findings. First, the results should be considered as preliminary because of a relatively small study sample, which may not provide sufficient power to pick up on interaction effects. Second, the intervention period of three weeks was relatively short. Mobile interventions may need a longer period of time to be able to promote long-term lifestyle changes rather than in the moment coping strategies (Ben-Zeev et al., 2014). However, one of the biggest advantages of working with ESM data is that through this collection technique subtle changes can be detected that might not be detected by standard questionnaire measures (Delespaul, 1995; Kimhy et al., 2012; Os et al., 2013). Third, multiple topics of symptoms, functioning and health-related behaviour were included in the feedback prompts. Because of this, prompts were not solely directed to symptoms or social behaviour. A stronger focus on feedback targeting social functioning may be more effective in improving functional outcome. Last, the study had no waitlist/treatment as usual control group; as such we cannot compare the results in the current study to treatment as usual (TAU) and are unable to differentiate between ESM without feedback and TAU influences.

5. Conclusion

This study suggests that mobile applications are feasible and incorporating personalised feedback prompts could be beneficial for individuals with SZ in reducing momentary psychotic symptoms. Decreased feelings of loneliness and questionnaire measured psychotic symptoms for all participants may be related to positive effects of monitoring of symptoms and experiences in daily life, study participation or a natural change for the better. Smartphone-based modalities with personalised feedback offer opportunities for simple and accessible interventions. They also offer a way to empower patients to take an active role in their mental health management. For future studies, it would be of particular interest to investigate whether the close integration of mobile interventions with personalised feedback in existing face-to-face treatments could further improve outcomes.

CRediT authorship contribution statement

Esther Hanssen: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing - original draft. Sanne Balvert: Formal analysis, Investigation, Project administration, Writing - review & editing. Margreet Oorschot: Conceptualization, Writing - review & editing. Karel Borkelmans: Software, Writing - review & editing. Jim van Os: Conceptualization, Writing - review & editing. Philippe Delespaul: Conceptualization, Writing - review & editing. Anne-Kathrin Fett: Conceptualization, Methodology, Supervision, Writing - review & editing.

Declaration of Competing Interest

None.

Acknowledgements

We would like to express our gratitude to all patients for providing valuable feedback during application development and study participation. We would like to thank Simpel for providing us with five smartphones including a 1GB, unrestricted calling and texting data plans in support of this research. We would like to thank Catherine van Zelst for providing feedback on the study concept. We are grateful for Natalie Castien, Suzanne Robberegt, Christi-Janne van As, Marlie Eemers, Niels den Daas and Reena Luijten for data collection and project support. This work was supported by funding of the Netherlands Organization for Scientific Research (NWO) VENI grant (451-13-035) and a NARSAD Young Investigator Grant from the Brain & Behaviour Research Foundation (24138) to Anne-Kathrin Fett. The funding sources had no involvement in the study design, collection, analysis or interpretation of the data, writing of the manuscript or the decision to submit the paper for publication.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2019.112695.

References

American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental

Disorders, 5th ed. American Psychiatric Association, Washington, DC.

Ben-Zeev, D., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., Mueser, K.T., 2014.

Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr. Bull. 40 (6), 1244-1253.

Ben-Zeev, D., Brian, R., Aschbrenner, K., Jonathan, G., Steingard, S., 2018. Video-based mobile health interventions for people with schizophrenia: bringing the “pocket therapist” to life. Psychiatr. Rehabil. J. 41 (1), 39.

Ben-Zeev, D., Wang, R., Abdullah, S., Brian, R., Scherer, E.A., Mistler, L.A., Hauser, M., Kane, J.M., Campbell, A., Choudhury, T., 2015. Mobile behavioral sensing for outpatients and inpatients with schizophrenia. Psychiatric Serv. 67 (5), 558-561.

Berry, K., Haddock, G., 2008. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychology and Psychotherapy: Theory, Research and Practice 81 (4), 419-436.

Billeke, P., Aboitiz, F., 2013. Social cognition in schizophrenia: from social stimuli processing to social engagement. Front. Psychiatry 4, 4.

Birchwood, M., Smith, J., Cochrane, R., Wetton, S., Copestake, S., 1990. The social functioning scale the development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br. J. Psychiatry 157 (6), 853-859.

Bowie, C.R., Leung, W.W., Reichenberg, A., McClure, M.M., Patterson, T.L., Heaton, R.K., Harvey, P.D., 2008. Predicting schizophrenia patients’ real-world behavior with specific neuropsychological and functional capacity measures. Biol. Psychiatry 63 (5), 505-511.

Bradshaw, W., Armour, M.P., Roseborough, D., 2007. Finding a place in the world: The experience of recovery from severe mental illness. Qualitative Social Work 6 (1), 27-47.

Burns, T., Patrick, D., 2007. Social functioning as an outcome measure in schizophrenia studies. Acta Psychiatr. Scand. 116 (6), 403-418.

Couture, S.M., Penn, D.L., Roberts, D.L., 2006. The functional significance of social cognition in schizophrenia: a review. Schizophr. Bull. 32 (suppl_1), S44-S63.

Delespaul, P.A., 1995. Assessing Schizophrenia in Daily life: The Experience Sampling Method. Maastricht University.

Depp, C.A., Perivoliotis, D., Holden, J., Dorr, J., Granholm, E.L., 2018. Single-Session mobile-augmented intervention in serious mental illness: a three-arm randomized controlled trial. Schizoph. Bull. 45 (4), 752-762.

Eglit, G.M., Palmer, B.W., A’verria, S.M., Tu, X., Jeste, D.V., 2018. Loneliness in schizophrenia: construct clarification, measurement, and clinical relevance. PLoS ONE 13 (3), e0194021.

Fett, A.-K.J., Viechtbauer, W., Dominguez, M.-d.-G., Penn, D.L., van Os, J., Krabbendam, L., 2011. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci. Biobehav. Rev. 35 (3), 573-588.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2015. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: a metaanalysis of cross-sectional studies. Schizophr. Bull. 42 (2), 448-455.

Firth, J., Torous, J., 2015. Smartphone apps for schizophrenia: a systematic review. JMIR Mhealth Uhealth 3 (4).

Garrido, G., Barrios, M., Penades, R., Enriquez, M., Garolera, M., Aragay, N., Pajares, M., Valles, V., Delgado, L., Alberni, J., 2013. Computer-assisted cognitive remediation therapy: cognition, self-esteem and quality of life in schizophrenia. Schizophr. Res. 150 (2), 563-569.

Granholm, E., Ben-Zeev, D., Link, P.C., Bradshaw, K.R., Holden, J.L., 2011. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull. 38 (3), 414-425.

Granholm, E., Loh, C., Swendsen, J., 2007. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophr. Bull. 34 (3), 507-514.

Hartmann, J.A., Wichers, M., Menne-Lothmann, C., Kramer, I., Viechtbauer, W., Peeters, F., Schruers, K.R., van Bemmel, A.L., Myin-Germeys, I., Delespaul, P., 2015.

Experience sampling-based personalized feedback and positive affect: a randomized controlled trial in depressed patients. PLoS ONE 10 (6), e0128095.

Heron, K.E., Smyth, J.M., 2010. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br. J. Health Psychol. 15 (1), 1-39.

Huppert, J.D., Weiss, K.A., Lim, R., Pratt, S., Smith, T.E., 2001. Quality of life in schizophrenia: contributions of anxiety and depression. Schizophr. Res. 51 (2), 171-180.

Kay, S.R., Fiszbein, A., Opfer, L.A., 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13 (2), 261.

Kimhy, D., Myin-Germeys, I., Palmier-Claus, J., Swendsen, J., 2012. Mobile assessment guide for research in schizophrenia and severe mental disorders. Schizophr. Bull. 38 (3), 386-395.

Konings, M., Bak, M., Hanssen, M., Van Os, J., Krabbendam, L., 2006. Validity and reliability of the CAPE: a self-report instrument for the measurement of psychotic experiences in the general population. Acta Psychiatr. Scand. 114 (1), 55-61.

Kramer, I., Simons, C.J., Hartmann, J.A., Menne-Lothmann, C., Viechtbauer, W., Peeters, F., Schruers, K., van Bemmel, A.L., Myin-Germeys, I., Delespaul, P., 2014. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 13 (1), 68-77.

Lim, M.H., Penn, D.L., 2018. Using Digital Technology in the Treatment of Schizophrenia. Oxford University Press, US.

Medalia, A., Saperstein, A., 2011. The role of motivation for treatment success. Schizophr. Bull. 37 (suppl_2), S122-S128.

Moran, E.K., Culbreth, A.J., Barch, D.M., 2017. Ecological momentary assessment of negative symptoms in schizophrenia: relationships to effort-based decision making and reinforcement learning. J. Abnorm. Psychol. 126 (1), 96.

Myin-Germeys, I., Birchwood, M., Kwapil, T., 2011. From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophr. Bull. 37 (2), 244-247.

Myin-Germeys, I., Klippel, A., Steinhart, H., Reininghaus, U., 2016. Ecological momentary interventions in psychiatry. Curr. Opin. Psychiatry 29 (4), 258-263.

Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., van Os, J., 2009. Experience sampling research in psychopathology: opening the black box of daily life. Psychol. Med. 39 (9), 1533-1547.

Os, J., Delespaul, P., Wigman, J., Myin-Germeys, I., Wichers, M., 2013. Beyond DSM and ICD: introducing “precision diagnosis” for psychiatry using momentary assessment technology. World Psychiatry 12 (2), 113-117.

Penn, D.L., Roberts, D.L., Combs, D., Sterne, A., 2007. Best practices: the development of the social cognition and interaction training program for schizophrenia spectrum disorders. Psychiatric services 58 (4), 449-451.

Penn, D.L., Spaulding, W., Reed, D., Sullivan, M., 1996. The relationship of social cognition to ward behavior in chronic schizophrenia. Schizophr. Res. 20 (3), 327-335.

Pinkham, A.E., Penn, D.L., 2006. Neurocognitive and social cognitive predictors of interpersonal skill in schizophrenia. Psychiatry Res. 143 (2-3), 167-178.

Pos, K., Franke, N., Smit, F., Wijnen, B.F., Staring, A.B., Van der Gaag, M., Meijer, C., de Haan, L., Velthorst, E., Schirmbeck, F., 2019. Cognitive behavioral therapy for social activation in recent-onset psychosis: randomized controlled trial. J. Consult. Clin. Psychol. 87 (2), 151.

Roberts, D.L., Velligan, D.I., 2012. Can social functioning in schizophrenia be improved through targeted social cognitive intervention?Rehabilitation Research and Practice 2012.

Robinson, D.G., Woerner, M.G., McMeniman, M., Mendelowitz, A., Bilder, R.M., 2004. Symptomatic and functional recovery from a first episode of schizophrenia or schizoaffective disorder. Am. J. Psychiatry 161 (3), 473-479.

Schlosser, D.A., Campellone, T.R., Truong, B., Etter, K., Vergani, S., Komaiko, K., Vinogradov, S., 2018. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr. Bull. 44 (5),

1010-1020.

Seppala, J.,De Vita, I., Jamsa, T., Miettunen, J., Isohanni, M., Rubinstein, K., Feldman, Y., Grasa, E., Corripio, I., Berdun, J., 2019. Mobile phone and wearable sensor-based mHealth approaches for psychiatric disorders and symptoms: systematic review. JMIR Ment. Health 6 (2), e9819.

Spaniel, F., Vohlidka, P., Hrdlicka, J., Kozeny, J., Novak, T., Motlova, L., Cermak, J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res. 98 (1-3), 312-317.

Stain, H.J., Galletly, C.A., Clark, S., Wilson, J., Killen, E.A., Anthes, L., Campbell, L.E., Hanlon, M.-.C., Harvey, C., 2012. Understanding the social costs of psychosis: the experience of adults affected by psychosis identified within the second Australian National Survey of Psychosis. Aust. N.Z. J. Psychiatry 46 (9), 879-889.

StataCorp, L., 2015. College Station, TX. STATA® software version 13.

Stefanis, N., Hanssen, M., Smirnis, N., Avramopoulos, D., Evdokimidis, I., Stefanis, C., Verdoux, H., Van Os, J., 2002. Evidence that three dimensions of psychosis have a distribution in the general population. Psychol. Med. 32 (2), 347-358.

Torous, J., Firth, J., 2016. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 3 (2), 100-102.

Velthorst, E., Fett, A.-K.J., Reichenberg, A., Perlman, G., van Os, J., Bromet, E.J., Kotov, R., 2016. The 20-year longitudinal trajectories of social functioning in individuals with psychotic disorders. Am. J. Psychiatry 174 (11), 1075-1085.

Visser, K.F., Esfahlani, F.Z., Sayama, H., Strauss, G.P., 2018. An ecological momentary assessment evaluation of emotion regulation abnormalities in schizophrenia. Psychol. Med. 48 (14), 2337-2345.

Wechsler, D., Coalson, D.L., Raiford, S.E., 1997. WAIS-III: Wechsler Adult Intelligence Scale. Psychological Corporation, San Antonio, TX.

Wykes, T., Steel, C., Everitt, B., Tarrier, N., 2008. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr. Bull. 34 (3), 523-537.

SCHRES-06532; No of Pages 6

Schizophrenia Research xxx (2015) xxx-xxx



Contents lists available at ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres

Computerised working-memory focused cognitive remediation therapy for psychosis A preliminary study

A. Hargreaves a,1> R. Dillon a-10 11, H. Anderson-Schmidta, A. Corvir a, B. Fitzmaurice a, M. Castorinac,d, I.H. Robertsonc,d, G. Donohoe a,b,c,*

a Department of Psychiatry, Trinity College Dublin, Dublin, Ireland

b CogGene group, School of Psychology & Center for Neuroimaging, Cognition & Genomics, National University of Ireland, Galway, Ireland

c Trinity College Institute of Neuroscience, Trinity College Dublin, Ireland

d School of Psychology, Trinity College Dublin, Ireland

ARTICLE INFO

ABSTRACT

Article history:

Received 24 March 2015

Received in revised form 31 August 2015

Accepted 1 September 2015

Available online xxxx

Background: Cognitive deficits are a core feature of schizophrenia and related psychotic disorders and are associated with decreased levels of functioning. Behavioural interventions have shown success in remediating these deficits; determining how best to maximise this benefit while minimising the cost is an important next step in optimising this intervention for clinical use.

Aims: To examine the effects of a novel working-memory focused cognitive remediation (CR) training on cognitive

Keywords:

Cognitive remediation therapy

Schizophrenia

Psychosis

Memory

Cognition

difficulties based on internet delivery of training and weekly telephone support.

Method: Participants with a diagnosis of psychosis (n = 56) underwent either 8 weeks of CR (approximately 20 h) or 8 weeks of treatment as usual (TAU). General cognitive ability, working memory and episodic memory were measured both pre and post intervention for all participants.

Results: In addition to improvements on trained working memory tasks, CR training was associated with significant improvements in two tests of verbal episodic memory. No association between CR and changes in general cognitive ability was observed. Effect sizes for statistically significant changes in memory were comparable to those reported in the literature based primarily on 1:1 training.

Conclusions: The cognitive benefits observed in this non-randomised preliminary study indicate that internet-based working memory training can be an effective cognitive remediation therapy. The successes and challenges of an internet-based treatment are discussed.

© 2015 Elsevier B.V. All rights reserved.

1. Introduction


A core feature of many psychotic disorders is poor cognitive performance. Deficits in cognition often predate the emergence of clinical symptoms, and then persist throughout the illness, strongly predicting functional outcome (Green et al., 2000; Wykes and Reeder, 2005). Because current antipsychotic medications do not adequately treat these deficits (Green, 1996; Fettet al., 2011), behaviour-based therapies designed to remediate cognitive deficits,anapproachknown as cognitive remediation' (CR), have become a significant focus of research.

CR has been used to refer to a number of interventions which seek to ameliorate difficulties with cognitive skills such as attention, memory, problem solving, information-processing speed, organisation, and planning. CR interventions differ widely in terms of method of administration (pen and paper versus computer), frequency of sessions, mode of administration (therapist-administered versus patient working alone) and method of training. Despite these differences, a meta-analysis by Wykes et al. (2011) based on more than two thousand participants found consistent evidence of cognitive gains associated with CR, yielding an average effect size of ~0.5 across the range of interventions considered. Importantly, these benefits are not confined to cognition; CR has also been shown to be associated with benefits to social and occupational functioning (Wykes et al., 2011).

Several questions about CR remain, however, including the costeffectiveness of the various approaches taken (Wykes, 2010). Even if the cost of CR compares favourably to currently used pharmacotherapy, the number of therapist hours involved are typically substantial, making delivery potentially problematic in standard clinical settings. Efforts to address this issue have included delivery of CR in a group setting (Medalia et al., 2001) and, recently, to make use of computer and/or internet-based approaches. While computerised approaches previously only permitted a one size fits all' approach, contemporary adaptive software enables task difficulty to be dynamically and automatically varied according to individual patient's response accuracy, and to changes in that response over time. This may permit patients the freedom to engage in training beyond the clinical setting and without the need of 1:1 support for each session. An important question for such e-health'

http://dx.doi.org/10.1016/j.schres.2015.09.004

0920-9964/© 2015 Elsevier B.V. All rights reserved.

2


A. Hargreaves et al. / Schizophrenia Research xxx (2015) xxx-xxx

Table 1

Number, age, chlorpromazine equivalent, gender and education of CR and TAU groups.

CR group

N = 22

TAU group

N = 26

t/x2

p

Psychosis subtype

SZ

N = 14

N = 19

SZA

N = 5

N=5

BP

N=1

N=1

PDNOS

N=2

N=1

Age

M (SD)

40.2 (8.8)

45.88(12.05)

-1.8

0.077

Chlorpromazine

M (SD)

495.7(515.7)

321.1(297.6)

1.1

0.278

equivalent

(mg/day)

Gender

M,F

17,5

18,8

0.503

0.478

Education: 1,2,3a

N in each respective category

7,9,6

10,7,9

5.74

0.57

a Education: 1, primary school education; 2, secondary school education; 3, post-secondary education.

initiatives is to determine patients' capacity to carry out such remotetraining and how much training support is required to adequately facilitate participation.

A further question for maximising the cost effectiveness of training compares the benefits of a more general versus a more specific (and potentially shorter-term) approach to the cognitive functions being trained in CR. For example, working memory (WM) deficits have been exclusively targeted both in patients with schizophrenia (Lawlor-Savage and Goghari, 2014; Takeuchi et al., 2010) and in patients with other disorders (Klingberg, 2010; Richmond et al., 2011; Shipstead et al., 2012). This approach is partly based on the hypothesis that WM improvements may benefit cognitive function more generally. In support of this hypothesis, training programmes that exclusively focused on working memory in non-schizophrenia populations have been associated with a transfer of benefits to other cognitive functions including attention, problemsolving, and to fluid intelligence (Lilienthal et al., 2013; Salminen et al., 2012; Kundu et al., 2013; Rudebeck et al., 2012; Jaeggi et al., 2010; Jaeggi et al., 2008). To date, only a few studies have exclusively targeted WM training (particularly auditory WM) in psychosis. Results have been promising, with WM training being associated with improvements in both verbal WM and general cognitive ability (Fisher et al., 2009; Hubacher et al., 2013; Subramaniam et al., 2014; Wexler et al., 2000; Haut et al., 2010). Whether a targeted approach such as this is more beneficial, either in terms of size or cost effectiveness of effect, however, remains uncertain (Wykes et al., 2011).

The aim of this study was to investigate the effectiveness of a novel 8week WM training programme, designed to be both ecologically valid and web-based, in patients with schizophrenia and related psychosis. In the preliminary phase of this study being reported here, we began to test this hypothesis by establishing whether cognitive performance improved in patients with psychosis who underwent CR training, and whether these changes differed from test-retest changes in clinically similarly patients receiving treatment as usual (TAU). Because WM is correlated with fluid intelligence, and WM training previously associated with gains in general cognitive ability (Jaeggi et al., 2010), we hypothesised that benefits to working memory capacity may lead to improvements in general cognitive ability.

2. Methods

2.1. CR participants

In the first stage of the study being reported here, 30 participants were recruited from community health teams from various sites across Ireland (Dublin, Wicklow, Sligo). Patients were referred by their local treatment teams following a series of presentations made about CR by the study team. All participants provided written informed consent and were interviewed using the Structured Clinical Interview for DSM-IV Axis 1 Disorders (SCID-I, First et al., 2002). DSM-IV diagnosis was established following a SCID interview and review of all available information interview, family or staff report, and chart review. Criteria for inclusion in the study were that participants were aged between 18 and 65 years, had a history of psychosis, were communitybased and clinically stable (in the opinion of the treating team), and were engaged in some activity (e.g. part time work or were attending a rehabilitation clinic for at least two days each week). Exclusion criteria included a history of organic impairment, head injury resulting in loss of consciousness, or drug abuse in the preceding 6 months.

2.1.1. Treatment as usual (TAU) participants

Immediately following the CR recruitment phase, a comparison group of 26 patient participants were ascertained from the same mental health services teams using the same inclusion and exclusion criteria as reported above. While these patients were therefore neither simultaneously collected nor randomly allocated, no differences in clinical or cognitive presentation were observed (see Table 1).

TAU consisted of multidisciplinary team input, including regular medical review, general psychosocial support from a community psychiatry nurse, and with additional inputs from occupational therapy and social work focusing on accommodation and occupation. Supportive group interventions (focusing on peer support and psychoeducation, and grounded in cognitive behavioural therapy principles) are available. While CBT for psychosis is also available on an individual basis, none of the patients in either the TAU or intervention conditions had received this in the six months before during or after participation in the study.

Table 2

Explanation of the 9 CR programme exercises and their relationship with the Baddeley Working Model.

Exercise

Baddeley Working Model

Faces snap

A series of pictures of faces are shown, when two in a row appear, have to click

VSSa & WM

Spaces snap

Square spaces are shown, when two appear in the same space, have to click

VSS & WM

Span colours

A grid of four square of four colours flashes in a certain sequence, have to remember that sequence

VSS

Span colours reverse

Same as above but sequence is entered in reverse

VSS & WM

Focus faces

A series of faces are shown, have to remember the last two faces

VSS & WM

Double snap

A combination of spaces snap and names snap (below)

VSS & PL & WMb

Names snap

A series of names are heard, have to remember the last two

PL & WM

Span numbers

A series of numbers are heard, have to type in all numbers at the end

PL

Span numbers reverse

Same as above but are entered in reverse (last number to first heard)

PL & WM

Maths mad

Number are called out and have to be added to 1-back (1, 4, 5 are called, 5 and 9 are entered)

PL & WM

Focus names

A series of names are displayed and the last two names need to be remembered

PL & WM

a Visuo-spatial sketchpad. b Phonological loop.

Please cite this article as: Hargreaves, A., et al., Computerised working-memory focused cognitive remediation therapy for psychosis A

preliminary study, Schizophr. Res. (2015), http://dx.doi.org/10.1016/j.schres.2015.09.004

A. Hargreaves etal./ Schizophrenia Research xxx (2015) xxx-xxx

2.2. CR programme

An online CR training programme specifically targeting WM, developed by our group, was employed (McAvinue et al., 2013). This programme, which was web-based targeted both auditory and visual WM modalities following Baddeley's (2000) model. Each of the 9 training tasks was designed to be, to at least some extent, ecologically valid by relating training to every-day tasks (e.g. remember the faces of people introduced to you at a party) (see Table 2 and Fig. 1). Prior to commencing training, computer access and training needs of participants were evaluated. If the participants did not have internet access, a laptop and internet dongle were provided as was any training required with accessing the training website and logging on.

The programme consisted of a mixture of psycho-education on the nature of working memory, strategy-based learning, and practice of nine working memory focused training exercises that were gradually introduced over a 5-week period, beginning with the easier exercises first. The exercises ranged from n-back tasks to classic digit-span tasks while maintaining real-life similarities by using real faces and scenarios. For example, on one task (the Faces task, see Fig. 2), participants are given the scenario that they are at a party and asked to pay attention to the people they meetat the party and are then later required to recollect the faces of the people they have been introduced to. During training, the exercises are adjusted in level of difficulty by changing the amount of information to be retained and the speed at which the information is presented, based on the participant's responses.

While the tasks themselves is primarily of the drill and practice variety, a strategy training component is also incorporated into the programme in both the online training and the weekly telephone support as follows: 1) before and after each practice session the applicability of individual WM tasks practised to real life situations are highlighted along with examples of how individual working memory strategies might be used in daily life; 2) during weekly phone calls, the therapist discussed with the participant how he or she applied these strategies, providing clarifications as needed and reinforcing the value of using these strategies. The strategies included techniques for exercise working memory day to day (e.g. attempting to remember phone numbers) and tips on how to organise information more effectively (e.g. learning how to use a mental blackboard). Details of the training exercises can be found in McAvinue et al. (2013).

Each participant was expected to practise the exercises for 30 to 40 min a day for 5 days a week (2 rest days of the patient's choice).

In total, each participant was required to complete 40 days of training (30/40 min of training each day) within a 12-week window. If participants wished to complete more than the required 30 min of exercises a day, they were encouraged to do so. As part of the programme, at the end of each session participants were given visual feedback via a graph of time in training and scores obtained so that they could track their individual progress.

Assistance in completing the CR programme involved 1) an initial start-up session to demonstrate how the programme worked and 2) weekly phone calls to monitor and encourage progress. The phone calls incorporated a problem-solving based approach and motivational coaching. Participants were encouraged to identify solutions to any difficulties encountered. For example if a day of training was missed, participants were asked to identify the cause of the missed day and how to overcome these. For example, if the cause lay with technical difficulties, help was immediately provided to rectify the problem. If the cause was motivational, a more therapeutic approach was taken where difficulties were identified and deconstructed and help was given to find a solution and provide further encouragement. Participants were also provided with a detailed instruction manual and logbook to mark and keep track of their progress. Participants' activity was monitored regularly online, with the researcher having access to individual exercise performance, quantity of exercises completed and time and date of exercise completion.

2.3. Assessment of outcome

All CR & TAU patients completed the following outcome measures before and immediately after training (after 12 weeks in the case of TAU).


4                                                          A. Hargreaves et al. / Schizophrenia Research xxx (2015) xxx-xxx

Episodic memory was assessed using the logical memory subtest from the Wechsler Memory Scale, 3rd edition (WMS-111; Wechsler, 1998). Working memory was assessed using letter number sequencing (LNS) fromWMS-III.

Our primary outcome measures consisted of total scores on both the episodic memory and working memory tasks administered. To estimate whether any changes observed on any of the primary outcome variables related to more generalised effects on cognition, we also included two measures related to general cognitive ability. These were the Similarities and Matrix Reasoning subtests from the Wechsler Abbreviated Scale of intelligence (Wechsler, 1999).

2.4. Statistical analysis

We approached our analysis of the effects of CR on our main outcome variables as follows: First, demographic and clinical characteristics between the CR and TAU groups were compared using independent t-tests (for continuous data) and chi-squared (for categorical data) in SPSS 22.0 (IBM, 2013). Continuous variables included age and chlorpromazine equivalents, while categorical variables included gender and education. For each test run, group (two levels: CR and TAU) was entered as the grouping variable and the demographic characteristic in question was entered as the test variable. Next, the cognitive function of CR and TAU groups was compared at baseline using a general linear model, analysis of covariance (ANCOVA), in SPSS version 22. This was to ascertain whether any cognitive differences existed between the two groups prior to intervention. The cognitive variable was entered as the dependent variable with treatment group (CR V TAU) entered as the independent variable. Finally, association between intervention and cognitive function, episodic memory and working memory was tested using a general linear model in SPSS version 22 (SPSS, 2014). A mixed analysis of covariance (ANCOVA, repeated measures) was undertaken for each cognitive outcome variable. Treatment group (CR v. TAU) was used as the independent variable, stage (baseline assessment v. follow-up assessment) was used as the within-subjects variable, and baseline assessment was used as a covariate. Effect sizes of statistically significant associations were determined using Cohen's d, calculated using the means and standard deviations of the CR and TAU groups at post-intervention assessment. Calculations were conducted using the effect size generator found at www.ClinTools.com.

3. Results

3.1. Sample characteristics

A total of 56 participants were recruited, with an average age of 43.5 years, of whom 35 (76%) were male (see Table 1). There was a 27% (n = 8) withdrawal rate from the CR group following recruitment; various reasons for withdrawal were reported, including discomfort with using a PC and not having sufficient free time to complete the training. No significant differences between the groups (CR group n = 22, TAU group n = 26) were observed in gender (x2 = 0.503, p = 0.478), medication dosage (chlorpromazine equivalents; t = 1.1, p = 0.278) or education level (x2 = 5.74, p = 0.57). For age, although not statistically significant, the TAU group trended towards being slightly older than the CR group (t= 1.8, p = 0.077 (see Table 1). Given the difference in age is 0.5 SD between the treatment groups, we ran the analysis with and without age as a covariate; no statistical difference in findings were observed between the two analyses. The results presented in Table 1 present the analyses run without co-varying for age.

3.2. Response to treatment intervention

A baseline comparison of the CR versus TAU groups revealed no significant differences between groups in cognitive performance, although a trend level difference in letter number sequencing performance was observed. At follow-up, by comparison, a number of significant differences emerged (see Table 3).

We next sought to establish whether any statistically significant differences were observed in the change from pre-assessment to post assessment. After co-varying for baseline differences in working memory, a trend level difference was observed between the CR group and the TAU group on working memory (letter number sequencing task; F = 3.14; p = .084). For episodic memory, patients in the CR group showed significantly greater improvements than the TAU group in episodic memory in both immediate and delayed recall paradigms (Logical memory 1: F = 9.78; p = .003; Logical memory 11: F = 6.69; p = .014; see Table 3). These differences represent medium sized effects according to Cohen's criteria (Logical memory I: Cohen's d = 0.46; Logical memory II: Cohen's d = 0.36). Finally, no difference was observed between treatment groups in general cognitive ability (WASI similarities: F = .062; p = .805, WASI Matrics: F = .071; p = .791, estimated full scale IQ(FSIQ): F = .338; p = .566). We observed no significant differences for the TAU group between pre and post measures on these tasks; by implication, the observed cognitive improvements in the CR group were unlikely to have been explained in terms of practice effects on these tasks.

As the amount of time participants spent engaged with the CR programme varied widely (mean minutes: 1030.2, SD minutes: 557; minimum 366.6 min, max 2592.3 min), we used Spearman's Rho to explore whether greater amounts of time engaged with the programme would result in greater cognitive improvements overall. No significant correlations were observed between total minutes and rate of change in performance following CR on any cognitive variable (all p > .05).

Only 5 subjects in total belonged to the Bipolar disorder (BD) and psychotic disorder not otherwise (PDNOS) specified groups (see Table 1). As SZ and BD differ in their profile of cognitive deficits to patients with SZ, we re-ran our analysis based on the SZ only cohort in order to compare results. Removing the BD and PDNOS patients from both the TAU and CR groups did not change the significance of the findings: the CR group continued to show significantly greater improvement over the TAU group in both immediate and delayed episodic memory (immediate: F = 10.32; p = 0.003; delayed: F = 6.07; p =

Table 3

Comparing CR intervention group to TAU group on neuropsychological performance pre- and post-intervention using analysis of covariance.

Pre-training

Post-training

CR

TAU

F

p

CR

TAU

F

p

Logical memory I

6.59 (3.34)

7.72 (3.7)

1.19

0.281

9.7 (3.37)

8.09 (3.63)

9.78

0.003

Logical memory II

7.4 (2.77)

7.8 (3.69)

0.164

0.687

9.38 (3.59)

8.09 (3.63)

6.69

0.014

Letter-number Sq

6.9 (3.29)

8.96 (3.94)

3.84

0.056

8.47 (2.52)

9.25 (3.89)

3.14

0.084

Estimated full scale IQ

96.1 (15.89)

90.73 (13.54)

1.1

0.301

99.05 (16.38)

91.18 (14.93)

0.338

0.566

Matrix reasoning

96.19 (19.6)

89.00 (13.19)

1.52

0.226

95.25 (22.01)

89.27 (17.12)

0.071

0.791

Similarities

97 (16.42)

93.53 (15.40)

0.41

0.526

96.8 (17.16)

93.18 (14.06)

0.062

0.805

A. Hargreaves etal./ Schizophrenia Research xxx (2015) xxx-xxx

4. Discussion

This study sought to ascertain the effectiveness of a novel 8-week WM training programme (completed within a 12-week window) on neuropsychological performance in patients with schizophrenia and related psychosis. Based on a comparison of patients receiving CR versus TAU, we observed 1) a trend-level association between CR training and improved performance on our primary outcome measure of working memory; 2) significant improvements in episodic memory as measured by the immediate and delayed conditions of the logical memory task. The effect sizes observed for these changes were moderate, consistent with estimates previously reported for CR (Wykes et al., 2011); and 3) the improvements in memory related tasks did not generalise to general cognitive functioning. These findings were unchanged when patients with psychotic disorders other than schizophrenia were removed from the analysis.

An important rationale for our study was determining the effectiveness and feasibility of a home-basedCR programme in which staff support was limited to telephone support ahead of carrying out a full randomised controlled trial. The findings from this preliminary study suggest that this approach is feasible: in stable community based patients, 74% were able to complete the programme using the telephone support available. Completers were defined by those having completed over 6 h (360 min) of the CR programme. Although still scarce, the literature evidences as little as 5 h of practice at sufficient intensity necessary to produced effects (McGurk et al., 2007; Wykes et al., 2011). These findings provide support for the approach taken, and echo other studies demonstrating the feasibility of home-basedcomputerised cognitive training programmes (Subramaniam et al., 2014; Ventura et al., 2013). As such, a randomised controlled trial of this intervention is warranted to establish the full benefits of this approach.

According to Wykes et al. (2011), one of the fundamental concepts of CR is that any cognitive benefits be generalisable across cognitive domains. A key finding of the current study is the transferability of cognitive benefit from WM training to episodic memory. This finding is in concordance with the literature where a positive association between WM and episodic memory performance in patients with psychosis has been reported (Kundu et al., 2013; Rudebeck et al., 2012). Two further cognitive domains have also been reported to be positively impacted by WM training; attentional ability (Kundu et al., 2013; Lilienthal et al., 2013) and fluid reasoning (Rudebeck et al., 2012; Jaeggi et al., 2008). Further investigation of these domains in relation to WM CR training would be desirable.

Unexpectedly, although WM specific CR proved significantly beneficial to episodic memory performance, training only resulted in trend level improvements in WM performance as measured by the letter number sequencing task when compared to TAU. This was despite the significant improvements on the nine working memory training tasks employed, and the significant change in letter number sequencing performance from pre to post training within the CR group. One explanation might be that many of the memory strategies taught in the CR programme were equally (if not more) relevant for episodic memory tasks than working memory tasks and that participants improved on the episodic memory tasks due to increased strategy use. However, this seems unlikely due to the CR tasks all being WM focused. A more likely explanation for this is our reliance on only one measure of working memory to assess outcome. This measure, which provided an index of verbal working memory, may not have been sufficiently sensitive to training related changes, given that our intervention focused on both visuospatial and auditory WM. A meta-analysis of WM training by Melby-Lervag and Hulme (2013) of 23 studies suggested that WM training produces reliable short- and long-term improvements in WM skills for visuospatial WM but only weaker short-term improvements for verbal WM. Omission of a visuo-spatial measure of working memory was a short coming of this study and will need to be addressed before the true effects of this training on working memory can be established.

Furthermore, we did not test whether the benefits observed persisted over a longer period of time or whether a certain amount of maintenance" training is required. This remains an area for investigation in future studies. Finally, and again in relation to measurements used, our reliance on a measure of general intelligence that did not specifically index fluid intelligence may similarly have resulted in a lack of sensitivity to detect additional training related changes.

As a preliminary study, a non-randomised comparison group (TAU) was employed, preventing us from making a definitive statement about the causality of the improvements. Limitations of the passive TAU comparison include the difference in study mediated social contact between the intervention and control groups. While the CR group was exposed to therapist contact during neuropsychological assessments, weekly phone calls and an initial in-person meeting to set up the programme, the control group met the therapist only for neuropsychological assessments. According to Ybarra et al. (2011), social interaction in and of itself may benefit cognition. In creating an active, rather than passive, control group, it may also be worth considering the inclusion of non CR specific computer games in order to ascertain whether it is the use of computers, rather than engagement with CR, that improves learning. A further limitation to the study is that assessors were not blinded to group condition, nor was group allocation randomly allocated. Blinding of raters has been associated with attenuated impacts observed for psychosocial interventions such as cognitive behavioural therapy as it minimises rater bias, and may have contributed to the effects observed in this study. To address these issues, a single-blind randomised controlled trial of our programme, employing an active placebo condition, and measuring training benefits on measures of real world functional outcomes as well as on cognition, is required and is currently underway.

In conclusion, the present study of a low-support computerised WM focused training programme, was associated with cognitive improvements in those patients who underwent training, with effects sizes comparable to those previously reported for other interventions. As noted, although promising these results are preliminary and require confirmation in an appropriately powered, randomised, blinded, controlled trial is required to confirm the validity of the cognitive benefits observed here.

Conflict of interest

All authors confirm that they have no conflict of interest in relation to this manuscript.

Contributors

Authors April Hargreaves and Rachael Dillon undertook the statistical analysis and wrote the manuscript.

Author Heike Anderson-Schmidt collected all data.

Authors Aiden Corvin and Brian Fitzmaurice coordinated patient recruitment to the study.

Authors Marco Castorina and Ian Robertson were involved with the development and management of the online computer training programme used in the research.

Author Gary Donohoe designed the study and wrote the protocol.

All authors contributed to and have approved the final manuscript.

Role of funding source

The funding source, The Health Research Board, Ireland (HRB1466) is responsible for all funding of this research.

Acknowledgements

The authors sincerely thank all patients would contributed to this study and all staff who facilitated their involvement. This research was generously supported by the Health Research Board, Ireland.

References

Baddeley, A., 2000. The episodic buffer: a new component of working memory? Trends

Cogn. Sci. (Regul. Ed.) 4, 417-423.

Fett, A.K., Viechtbauer, W., Dominguez, M.D., Penn, D.L., van Os, J., Krabbendam, L., 2011. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci. Biobehav. Rev. 35, 573-588.

First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., 2002. Structured Clinical Interview for DSM-IV-TR Axis I Disorders. Research Version, Patient Edition. (SCID-I/P) New York: Biometrics Research, New York State Psychiatric Institute (November).

6                                                          A. Hargreaves etal./ Schizophrenia Research xxx (2015) xxx-xxx

Fisher, M., Holland, C., Merzenich, M., Vinogradov, S., 2009. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am. J. Psychiatry 166, 805-811.

Green, M.F., 1996. What are the functional consequences of neurocognitive deficits in schizophrenia? Am. J. Psychiatry 153,321-330.

Green, M.F., Kern, R.S., Braff, D.L., Mintz, J., 2000. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the "right stuff"? Schizophr Bull. 26 (1),119-136.

Haut, K.M., Lim, K.O., MacDonald, A., 2010. Prefrontal cortical changes following cognitive training in patients with chronic schizophrenia: effects of practice, generalization, and specificity. Neuropsychopharmacology 35, 1850-1859.

Hubacher, M., Weiland, M., Calabrese, P., Stoppe, G., Stocklin, M., Fischer-Barnicol, D., Opwis, K., Penner, I., 2013. Working memory training in patients with chronic schizophrenia: a pilot study. PsychiatryJ. 2013,154867.

IBM Corp, 2013. IBM SPSS Statistics for Windows, Version 22.0. IBM Corp, Armonk, NY.

Jaeggi, S.M., Buschkuehl, M., Jonides, J., Perrig, W.J., 2008. Improving fluid intelligence with training on working memory. Proc. Natl. Acad. Sci. U. S. A. 105, 6829-6833.

Jaeggi, S.M., Studer-Luethi, B., Buschkuehl, M., Su, Y.,Jonides,J., Perrig, W.J., 2010. The relationship between n-back performance and matrix reasoning-implications for training and transfer. Intelligence 38, 625-635.

Klingberg, T., 2010. Training and plasticity of working memory. Trends Cogn. Sci. 14 (7), 317-324 (Jul).

Kundu, B., Sutterer, D.W., Emrich, S.M., Postle, B.R., 2013. Strengthened effective connectivity underlies transfer of working memory training to tests of short-term memory and attention. J. Neurosci. 33, 8705-8715.

Lawlor-Savage, L., Goghari, V.M., 2014. Review: working memory training in schizophrenia and healthy populations. Behav. Sci. 4, 301-319.

Lilienthal, L., Tamez, E., Shelton, J.T., Myerson, J., Hale, S., 2013. Dual n-back training increases the capacity of the focus of attention. Psychon. Bull. Rev. 20, 135-141.

McAvinue, L.P., Golemme, M., Castorina, M., Tatti, E., Pigni, F.M., Salomone, S., Brennan, S., Robertson, I.H., 2013. An evaluation of a working memory training scheme in older adults. Front. Aging Neurosci. 5, 20 (May 23).

McGurk, S.R., Twamley, E.W., Sitzer, D.I., McHugo, G.J., Mueser, K.T., 2007. A meta-analysis of cognitive remediation in schizophrenia. Am. J. Psychiatry. 164 (12), 1791-1802.

Medalia, A., Revheim, N., Casey, M., 2001. The remediation of problem-solving skills in schizophrenia. Schizophr. Bull. 27, 259-267.

Melby-Lervag, M., Hulme, C., 2013. Is working memory training effective? A meta-analytic review. Dev. Psychol. 49 (2), 270-291 (Feb).

Richmond, L.L., Morrison, A.B., Chein,J.M., Olson, I.R., 2011. Working memory training and transfer in older adults. Psychol. Aging 26, 813-822.

Rudebeck, S.R., Bor, D., Ormond, A., O'Reilly,J.X., Lee, A.C., 2012. A potential spatial working memory training task to improve both episodic memory and fluid intelligence. PLoS One 7, e50431.

Salminen, T., Strobach, T., Schubert, T., 2012. On the impacts of working memory training on executive functioning. Front. Hum. Neurosci. 6, 166.

Shipstead, Z., Redick, T.S., Engle, R.W., 2012. Is working memory training effective? Psychol. Bull. 138, 628-65410.

SPSS, 2014. SPSS 22.0 Command Syntax Reference. SPSS Inc., Chicago.

Subramaniam, K., Luks, T., Garrett, C., Chung, C., Fisher, M., Nagarajan, S., Vinogradov, S., 2014. Intensive cognitive training in schizophrenia enhances working memory and associated prefrontal cortical efficiency in a manner that drives long-term functional gains. NeuroImage 99, 281-292.

Takeuchi, H., Taki, Y., Kawashima, R., 2010. Effects of working memory training on cognitive functions and neural systems. Rev. Neurosci. 21 (6), 427-449.

Ventura, J., Wilson, S.A., Wood, R.C., Hellemann, G.S., 2013. Cognitive training at home in schizophrenia is feasible. Schizophr. Res. 143 (2-3), 397-398 (Feb).

Wechsler, D., 1998. Wechsler Memory Scale, Third edition (WMS-III). The Psychological Corporation, New York.

Wechsler, D., 1999. Wechsler Abbreviated Scale of Intelligence. The Psychological Corporation: Harcourt Brace & Company, New York, NY.

Wexler, B.E., Anderson, M., Fulbright, R.K., Gore, J.C., 2000. Preliminary evidence of improved verbal working memory performance and normalization of task-related frontal lobe activation in schizophrenia following cognitive exercises. Am. J. Psychiatry 157,1694-1697.

Wykes, T., Reeder, C., 2005. Cognitive Remediation Therapy for Schizophrenia, Theory and Practice. Routledge, East Sussex.

Wykes, T., Huddy, V., Cellard, C., McGurk, S.R., Czobor, P., 2011. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am. J. Psychiatry 168 (5),472-485 (May).

Wykes, T., 2010. Cognitive remediation therapy needs funding. Nature 468 (7321), 165-166 (Nov 11).

Ybarra, O., Winkielman, P., Yeh, I., Burnstein, E., Kavanagh, L., 2011. Friends (and sometimes enemies) with cognitive benefits. What types of social interactions boost executive functioning? Soc. Psychol. Personal. Sci. 2 (3), 253-261 (May).


Contents lists available at ScienceDirect

Psychiatry Research

journal homepage: www.elsevier.com/locate/psychres

Using telehealth to augment an intensive case monitoring program in veterans with schizophrenia and suicidal ideation: A pilot trial



John Kasckowa,b,c,12, Susan Zickmundc,d, John Gurklisa, James Luther a, Lauren Foxa, Melissa Taylor e, Ira Richmond6, Gretchen L Haasa,b

a MIRECC and Behavioral Health, VA Pittsburgh Health Care System, Pittsburgh, PA 15240, United States

b Western Psychiatric Institute and Clinic, University of Pittsburgh Medical Center, Pittsburgh, PA 15213, United States

c VA Pittsburgh Center for Health and Equity Promotion, Pittsburgh, PA 15240, United States

d Department of Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA 15213, United States

e VA Pittsburgh Patient Care Services, VA Pittsburgh Health Care System, Pittsburgh, PA 15240, United States

ARTICLE INFO


ABSTRACT


Article history:

Received 22 July 2015

Received in revised form

10 November 2015

Accepted 21 February 2016

Available online 24 February 2016

Keywords:

Schizophrenia

Telehealth

Suicide


Veterans with schizophrenia admitted for suicidal ideation were recruited into a post-discharge program consisting of Intensive Case Monitoring (ICM) with daily monitoring with the Health Buddy (HB; experimental group) or ICM alone (control group). This study tested the feasibility of the telehealth monitoring intervention in this population. Secondly, we determined whether augmentation of ICM with our intervention for 3 months would result in a reduction in suicidal ideation. Twenty of 25 telehealth participants could set up the device. Monthly adherence for telehealth participants was > 80%. A qualitative analysis of endpoint surveys revealed that the majority of participants had positive responses. In both groups, there were improvements in Beck Scale for Suicidal Ideation (BSS) scores at endpoint relative to baseline. No group differences were present with survival analysis when using remission (i.e., BSS score = 0) as the outcome; however, in a subgroup with a history of suicide attempt, there was a trend (p = .093) for a higher rate of remission for those in the HB condition. In conclusion, telehealth monitoring for this population appears to be feasible for those who are able to start using the system. The pilot data obtained should help investigators design better telehealth interventions for this population.

Published by Elsevier Ireland Ltd.

Suicide is a leading cause of premature death among people with schizophrenia (Kasckow et al., 2011). Many patients with schizophrenia who are hospitalized for suicidal ideation or attempt require post-hospitalization follow-up (While et al., 2012). This is particularly important during the first 3 months post-discharge when patients are at highest risk for suicide (e.g., While et al., 2012). To have the greatest impact on suicide prevention during this high-risk time, the American Psychiatric Association has recommended intensive monitoring (American Psychiatric Association, 2006). Most monitoring strategies for suicide prevention rely on in-person or telephone contact with a clinician or health care provider (Valenstein et al., 2009).

Veterans are a select U.S. subpopulation experiencing greater than average increases in suicide rates. Suicide is a leading cause of death in Veterans (Bruce 2010). In 2010, it was estimated that up to 22 Veterans commit suicide each day. The VA has responded to escalating suicide rates since 2008 by including enhanced mandated monitoring, the creation of high risk suicide lists and a 24 h hot line along with placement of suicide prevention coordinators. When inpatient Veterans are placed on the high risk list, they undergo intensive monitoring. Upon discharge following a hospitalization for suicidal behavior, they are monitored for at least once a week for the first month and then at least monthly for the remaining 2 months after hospital discharge for suicidal behavior. Since this monitoring policy was implemented, suicides among Veterans who receive VHA services have decreased somewhat. However, they still remain elevated at an unacceptable level. For instance, Hoffmire et al. (2015) indicated that the number of observed veteran suicides is still approximately 20% higher than that which occurred in 2000. Thus, additional strategies for intervention are warranted.

Our research team has developed a telehealth monitoring system for suicidal patients with schizophrenia using the Health Buddy©, a telephone device that facilitates symptom assessment and patient-staff communication between visits. This telehealth intervention involves augmenting an Intensive Case Monitoring (ICM) program with daily use of the Health Buddy© system. ICM included weekly face-to-face meetings at the hospital clinic plus twice weekly phone calls in addition to standard VA monitoring. These visits and phone calls included assessment using the Patient Health Questionnaire (PHQ9; Kroenke et al., 2001) and the Beck Scale for Suicidal Ideation (BSS; Beck et al., 1979).

http://dx.doi.org/10.1016jj.psychres.2016.02.049

0165-1781/Published by Elsevier Ireland Ltd.


The purpose of the current study was to first test the feasibility of the telehealth monitoring intervention for suicidal behavior in this population of Veterans with schizophrenia or schizoaffective disorder. To address this, we asked whether participants would be willing to engage with the technology-based intervention, and continue to use it over a 3 month period. The secondary purpose was to assess with a random assignment trial, whether augmentation of ICM with our intervention would result in a significant reduction in suicidal ideation relative to a group that received only ICM.

We had previously reported that when augmenting the Health Buddy telehealth system with VA Treatment as Usual, there were improvements in suicidal ideation in at-risk Veterans with schizophrenia and suicidal ideation (Kasckow et al., 2015). There have been very few studies examining the use of telehealth for monitoring Veterans at risk for suicide. With the current study, our control condition, i.e., intensive case monitoring (ICM) was different and we obtained different outcomes compared to what we noted previously. Research in this area is expected to grow in the future and investigators will need guidance in the design of such approaches. Thus, it is important that we report our findings with this different control group to help provide guidance.

All procedures were approved by the Institutional Review Board of the VA Pittsburgh Health Care System. The research team assessed recently admitted inpatients > 18 years old for a diagnosis of schizophrenia/schizoaffective disorder and recent suicidal ideation; if an eligible patient was found, the clinician would be asked to refer the patient and ask the patient to sign a HIPAA form to provide permission for further screening and contact by the research team. If signed, the research team would discuss the protocol with the patient. If the patient was interested, s/he was invited to participate in an informed consent process wherein written informed consent was obtained. Following the consent procedures, the patients were further screened for the following inclusion criteria: a Mini Mental Status score > 20 (Folstein et al., 1975); lack of a medical disorder which could influence diagnostic decisions, safety, and/or anticipated adherence. An example of a participant who had an exclusionary medical illness was an individual with recurrent hematemesis due to esophageal varices. Another example was an

Table 1

Sociodemographic measures by treatment.

Measure

Total (N=51)

HB (N = 25)

ICM (N=26)

p-Value

Age (years)

51.1 (11.3)

51.0 (11.7)

51.2 (11.1)

0.838a

Race

0.136b

Black

17 (34.0)

6 (24.0)

11 (44.0)

White

33 (66.0)

19 (76.0)

14 (56.0)

Education (in years)

12.6 (1.80)

12.7 (2.29)

12.5 (1.12)

0.734a

Marital status

0.287c

Married/living with a

8 (15.7)

4 (16.0)

4 (15.4)

partner

Separated/divorced

15 (29.4)

10 (40.0)

5 (19.2)

Never married

23 (45.1)

8 (32.0)

14 (57.7)

Widowed

5 (9.8)

3 (12.0)

2 (7.7)

Note: ICM=Intensive Case Monitoring group; HB=Telehealth+ICM group.

a Mann-Whitney test.

b Chi-squared test.

c Fisher's exact. Values represent means (standard deviations) for continuous variables or N (percentages) for categorical measures. individual with motor dexterity problems due to neurologic diseases which prevented them from effectively using the telehealth device.

Other inclusionary criteria included a score > 0 on item 4 and/or 5 of the BSS which respectively assesses active suicidal ideation and passive suicidal ideation, and a score > 8on the 17-item Hamilton Depression Rating Scale (Hamilton, 1960). Patients also needed to have a land-line telephone. At baseline, we also obtained demographic and other clinical data using the Calgary Scale for Depressive Symptoms (Addington et al., 1992), the Scales for Assessment of Positive Symptoms (Andreasen et al., 1995) and Negative Symptoms (Andreasen, 1992) [see Tables 1 and 2]. Recruitment occurred from 2/2008 to 9/2011.

Participants were randomized to ICM alone (control condition) or ICM with daily Health Buddy© monitoring (experimental condition). All participants received usual mental health care. This included weekly assessments with the BSS (Beck et al., 1979) and Personal Health Questionnaire 9 (PHQ9; Kroenke et al., 2001), which were administered face-to-face and also twice weekly on the phone by nurses; nurses' inter-rater reliability intra class correlation was > 0.9. The Beck scale was the standard scale with 19 items and a range of 0 - 38 (Beck et al. 1979). Face-to-face assessments decreased to every other week if BSS scores were 0 for 4 weeks and then to monthly with once weekly phone assessments if BSS scores were 0 for another 6 weeks. Face to face assessments also included the Calgary Scale for Depressive Symptom (Addington et al. 1992), Hamilton Depression Rating Scale (Hamilton, 1960) and the Scales for Assessment of Positive Symptoms and Negative Symptoms (Andreasen et al. 1995).

Participants randomized to the telehealth group were provided the Health Buddy device upon discharge from the inpatient psychiatric unit. They were provided instructions on how to use the device. This involved standard procedures (provided by Bosch HealthCare) to ensure that participants were able to understand what power connections were needed and that participants were able to press the appropriate buttons to start the dialogues. In addition, staff ensured that participants could answer questions appropriately by pushing the buttons on the device, end the daily sessions and understand how information is sent to the care providers. They would also explain to participants that they should contact support staff about any equipment/power failures or any other questions.

Daily telehealth monitoring included queries for participants about suicide, depressive symptoms and medication adherence utilizing dialogues provided originally by Health Hero, the company which originally marketed the Health Buddy©. Bosch Healthcare now administers the Health Buddy system in the Department of Veterans Affairs. The Health Buddy© connected to the participant's land-line telephone; each day, for 10-15 min, a participant answered questions by pushing one of the buttons on the device. Responses were then electronically transferred to the hospital daily and read by staff within 4 h of transfer. The reason for the four hour time frame was to balance 2 factors: (1) minimizing the time in which a potentially high risk response would be transmitted by the patient and subsequently read by a clinician and (2) practical issues with regard to how frequently clinical staff could monitor the website.

The HB dialogues provided daily psychoeducational support and would assist participants in deciding whether they should contact their clinician with worsening symptoms. Participants would also be provided with the phone number for the crisis line if they stated they had suicidal intent and/or plan. Copies of the dialogues are available upon request. Furthermore, participants returned the telehealth devices at the end of the study.

If participants did not download responses within 24 h since the last time this was done, they would be contacted by staff to ensure that they were safe and to remind them to continue to use the device. In addition, staff would immediately contact patients if they responded yes' to a question that inquired about suicidal behavior. In this case, staff members would assess the situation and decide whether (1) no action was needed; (2) whether the participant needed to come in soon to see their clinician (if there was not an appointment scheduled in the near future); (3) whether participants needed to come to the emergency room or; (4) if urgent assessment was needed and if the participant was unwilling to come in, whether the police needed to go to the participants' home for further evaluation.

After they completed the study, participants completed a structured survey with the option of including open-ended responses. The survey asked them to write their assessment (i.e., judgement) of the telehealth intervention, including its strengths and how it could be improved. For the analysis of the participants' statements (i.e., actual words used), all comments by participants were linked together based on their study ID. Two trained qualitative coders judged whether each participant was: (1) generally positive about the program, (2) generally negative, or (3) whether no assessment of the program could be made. Each coder judged the patients' statements independently and then compared the results. There was a single disagreement between the coders (one selecting a positive' rating while the other assessed it as cannot judge'), which was resolved through discussion. The positive' rating was chosen as the final judgment. The overall inter-coder reliability kappa statistic was 0.857 which is what Landis and Koch (1977) has described as near perfect' reliability. The senior qualitative analyst [SZ] then

Table 2

Baseline clinical measures by treatment.

Measure

Total (N=51)

HB (N-25j

ICM (N=26)

p-Value

Beck scale for suicidal ideation (SSI) score

10.3 (7.23)

9.80 (6.15)

10.7 (8.24)

0.477

17 item Hamilton depression rating scale

16.0 (6.53)

17.0 (5.47)

15.2 (7.31)

0.347

Calgary depression rating scale

12.2 (5.13)

12.9 (4.74)

11.6 (5.48)

0.381

Mini mental status exam

26.7 (2.25)

27.1 (2.44)

26.2 (2.01)

0.092

Scale for assessment of positive symptoms (global)

5.36 (3.06)

5.00 (3.37)

5.71 (2.76)

0.596

Scale for assessment of negative symptoms (global)

11.40 (2.30)

11.40 (2.30)

11.20 (2.30)

0.728

Note: All comparisons were made by Mann-Whitney test. Values represent means (standard deviations) for all measures. ICM = Intensive Case Monitoring Group; HB = Telehealth + ICM Group.

used qualitative content analysis based on Miller and Crabtree (1992) known as the editing styleto highlight reasons for the positive or negative statement. This approach involved developing an iterative codebook based on a close reading of the text. The purpose of this was to better understand the reasons participants provided their judgements which are referred to as assessments' in the text.

Continuous measures were expressed as means and standard deviations. Tests of association included Student's t or Mann-Whitney tests. Categorical measures

were expressed by frequency and percentage distributions; tests of association included chi-square or the Fisher's exact test if cell frequencies were small. We examined changes in suicidal ideation between groups by survival analysis with either time to remission, i.e., BSS score=0 or as % response (i.e., > 50% change in BSS scores) as the endpoint. We also examined whether there were differences in the other outcome measures, i.e., scores from the Calgary Depression Rating Scale, the 17 item Hamilton Depression Rating Scale, Scale for Positive Symptoms and Scale for Negative Symptoms. With each scale, scores were fit with a repeated measures mixed model which included random intercepts and an unstructured covariance matrix. To account for the tendency of scores to increase toward the end of the study, a quadratic term was added to each model.


A total of 1628 inpatients had been screened for the trial. Twenty five patients were randomized to the telehealth experimental group and 26 were randomized to the ICM group. Fig. 1 displays the recruitment flow chart for the study. Tables 1 and 2 show baseline demographic and clinical characteristics and indicate that there were no significant group differences. Not shown are the gender differences; there were 2 female control participants and 1 female experimental participant. Furthermore, 24/25 telehealth and 21/26 control participants had a lifetime history of substance abuse/dependence; and 7/25 telehealth and 9/26 control participants had substance abuse/dependence a month prior to screening. The two groups did not differ in terms of frequency of substance abuse/dependence conditions.

To address feasibility, 5 out of 25 telehealth participants never set up the system. One patient's landlord did not allow him to use the technology on the premises. The second patient was too disorganized and cognitively impaired (MMSE score=21) and the third found out upon returning home that he had a phone company debt which he could not pay. The fourth participant relapsed to substance dependence and the fifth participant realized that he had transportation problems and would not be able to make it in for the face-to-face assessments and decided to withdraw soon after randomization. In addition, of the 20 who were able to start the system, 4 required assistance, i.e., staff visited patients' residences to help with set-up.

Monthly adherence for participants was as follows: month 1: 83% (n=20); month 2: 92% (n = 19) and month 3: 89% (n=15). Rates were calculated monthly by adding for each participant the number of days they filled out the questions divided by the number of days they were in the study that month. In month 1, 1 participant dropped out at week 3. In month 2, 4 more dropped out at weeks 6, 6, 7 and 7 and in month 3, 1 more dropped out at week 10. To further address feasibility, we identified two adherence patterns among the 20 telehealth participants who started using the system over the 3 months: We defined a HIGHLY adherentgroup (n = 11) as those who exhibited average daily adherence rates > 80%. We defined a MODERATELY adherent” group (n = 9) as participants who exhibited at least 1 month of daily adherence which was less than 80%. Interestingly, among the moderately adherent participants, post-hoc review of data from face-to-face interviews revealed that factors associated with moderate adherence included: exacerbation of depression (n=2), technical problems (n=4), a disruptive home environment (n=2), or unknown (n = 1).

Fourteen of the telehealth participants completed open-ended responses at the end of the written survey aimed at evaluating the telehealth intervention. A total of 44 statements were written by participants. From the 14 surveys, 17 of the responses were judged to be positive, 7 negative, and 20 provided no information to judge either way (e.g., statements such as no comment).

For those with negative statements, concerns over the telehealth intervention focused on the limitations perceived in symptom management, as well as its impersonal nature. For example, one person expressed great concern over the im-personalized treatment that came from communicating with a computer. This participant wanted [t]he human equation.... More emphasis on the whole person. not a subject of a study to be analyzed, not automation but as a real live man with deep psychological problems.The BSS scores had improved in this individual over the 3 month intervention period. Another participant expressed frustration about the lack of symptom abatement: Some days when I got confronted with suicidal questions, it would make me feel frustrated since the symptoms did not get better.For this individual the BSS scores got worse over the 3 month intervention period.

For those with positive statements, the telehealth intervention was praised for its ability to instill hope. One participant noted: It really helped me a lot when I had bad days, it gave me hope.” Patients described having a sense of being listened to and they noted that the program was akin to discussing problems with a medical provider. As one participant wrote about the program: It was helpful and straightforward. It was like talking to a doctor on a daily basis; the 1st month I did not think it would help but I changed my mind.Participants also described the program as effective in decreasing their suicidal thoughts. One participant concluded that it made him aware of the undesirability of taking my own life.drummed into my mind and psyche the desire and motivation to keep on living and trying, praying and believing in God as my source.Other participants described the program as effective in terms of improving their medication adherence, and symptom reduction for anxiety and depression.

Both groups exhibited improvements in suicidal ideation. At baseline, the mean (+/- standard deviation) HB BSS score was 9.8 ( + 6.15) and, at endpoint, the mean score was 2.44 ( + 5.52). For the control group, the mean baseline BSS score was 10.7 (+ 8.24) and, at endpoint, the mean score was 2.88 ( + 6.71). Using survival analysis, we did not detect any group differences when examining time to remission, defined as having a BSS score=0 nor when examining % response (i.e. those with > 50% change in BSS scores). For the subgroup of participants who had a lifetime history of suicide attempt (i.e., excluding those who had experienced ideation only), we found a trend for a higher rate of remission at

In addition, we used repeated measures regression analysis to examine whether there were changes in the clinical measures in the ICM+HB group vs the ICM group; this included scores in Calgary Depression Rating Scale, Hamilton Depression Rating Scale, Scale for Positive Symptoms and Scale for Negative Symptoms. No differences were detected between groups.

Our telehealth intervention has as its primary purpose daily monitoring with early detection of suicide risk so that clinicians know when to intervene. It combines engagement, monitoring and early intervention. Secondarily, the intervention provides supportive coaching and psychoeducation. Our pilot findings suggest that the use of our telehealth monitoring system is feasible in monitoring post-discharge suicide risk in this population. The population we studied is considered to be at high risk for nonadherence (Daniels et al., 2014).

Twenty out of 25 participants could set up the system. Of those who set up the system, the majority of participants in the ICM+HB group completed the 3 month protocol. Furthermore of the 20 who started using the system, only 4 dropped out within the 3 month protocol. One dropped out because of a lack of interest. Another one moved to another town. The other 2 dropped out because of circumstances beyond their control: 1) transportation problems and 2) incarceration. The overall adherence for the remaining 20 participants for logging in daily to the telehealth system exceeded 80% for each of the 3 months while they were in the study. This is consistent with what was reported previously for the same population of Veterans (Kasckow et al. 2015).

Telehealth monitoring has been shown to be acceptable for patients with schizophrenia using telehealth modalities other than the Health Buddy system (Kasckow et al. 2014). The current study also assessed acceptability; acceptability has not been reported previously in studies involving suicide monitoring with this population (Kasckow et al. 2015). Negative statements indicated that patients at times became frustrated when communicating with a machine rather than with a human. Despite there being some negative statements, the analysis of our qualitative data determined that the majority of the participants who responded to the survey had positive responses. The positive statements indicated that the system helped instill hope and that the system was helpful and easy to use. These statements have also been useful for further improving the telehealth system.

Although each group exhibited substantial improvement in endpoint vs baseline BSS scores, there were no statistically significant group differences. This may be due to the relatively intensive nature of the control condition which both groups received (i.e., two calls from the research nurse per week and one face-to-face visit per week); patients assigned to the control condition exhibited a marked treatment effect by itselfan outcome which may have left little room for further improvement when telehealth was used as an adjunct to this treatment. We implemented these intensive measures into our design because of safety concerns. In retrospect, while we realize that this control condition helped to enhance safety, it also consisted of a level of intervention considerably more intensive than typically provided in outpatient mental health settings. This issue reflects a common problem with research involving suicidal participants, i.e., finding the correct balance between achieving sufficient scientific rigor vs maintaining high ethical standards in order to maximize safety with a high risk population (Reynolds et al., 2001). As investigators design more studies in the future involving telehealth and populations at risk for suicide, establishing the appropriate control will be important. Human Subjects oversight boards tend to err on the side of caution and act in a conservative manner when deciding which control condition would be appropriate. Our findings thus provide important guidance in this regard.

In the current trial, a weak signal was detected when assessing suicidal ideation in participants with higher risk, i.e., those with a history of suicide attempt. This suggests that the intervention may have a stronger impact on those with a history of a suicide attempt. We also determined that the number needed to treat' for reaching a BSS score of 0 by week 28 with telehealth was 6 in the subgroup of participants with a history of suicide attempts. Future trials with a sufficiently powered sample size may help verify that significant improvements in suicidal ideation with telehealth can be demonstrated in participants with a history of a suicide attempt.

Study limitations included lack of double blinding; lack of double blinding may have caused information bias. The high adherence rates could have been due to selection bias since participants were willing to consent to a trial; this may have affected external validity. The sample size was small and included only veterans from a single urban eastern US site. There were 37 patients who screened as eligible who did not sign the HIPAA form; this could have caused a selection bias. The baseline scores were not marked. This is due to the fact that our inclusion criteria only required participants to have a screening score of only 1 on item 4 and/or 5 on the BSS. The purpose of requiring a lower score was to maximize recruitment. It is not known whether having a sample of higher risk participants with higher BSS scores would have been associated with significant differences in BSS scores. Future studies will be aimed at answering that question.

Telehealth technology allows patients and clinical staff ease and efficiency of communication. This is important given that patients with schizophrenia often become isolated and, if nonadherent to appointments, can become out of reach of the outpatient treatment team. Our findings add to the literature which supports that self monitoring is an important component of disease management. For instance, self monitoring has been shown to be important in improving outcomes with weight loss (Burke et al., 2011) and cardiovascular diseases (Heneghan et al., 2006). Application of a telehealth clinical monitoring system holds promise in efforts to monitor suicide risk. Progress has been made with the use of telephone-, video- and internet-based modalities in the treatment of patients with schizophrenia (Kasckow et al., 2014). Our current telehealth system represents another approach.

Acknowledgment

Funded by the VISN 4 MIRECC and a VISN 4 CPPF award. The contents do not represent the views of the US Government or the Department of Veterans Affairs of the US Government. Dr. Kasck-ow has received assistance from Bosch Health Care for the software and transmission costs associated with this project.

References

Addington, D., Addington, J., Maticka-Tyndale, E., Joyce, J., 1992. Reliability and validity of a depression rating scale for schizophrenics. Schizophr. Res. 6, 201-208.

American Psychiatric Association, 2006. Practice Guidelines for the Treatment of Psychotic Disorders. Psychiatry Online, Arlington, VA.

Andreasen, N.C., 1982. Negative symptoms in schizophrenia Definition and reliability. Arch Gen Psychiatry 39 (7), 784-788.

Andreasen, N.C., Arndt, S., Miller, D., Flaum, M., Nopoulos, P., 1995. Correlational studies of the scale for the assessment of negative symptoms and the scale for the assessment of positive symptoms: an overview and update. Psychopathology 28, 7-17.

Beck, A.T., Kovacs, M., Weissman, A., 1979. Assessment of suicidal intention: the scale for suicide ideation. J. Consult. Clin. Psychol. 47, 343-352.

Burke, L.E., Wang,J., Sevick, M.A., 2011. Self-monitoring in weight loss: a systematic review of the literature. J. Am Diet. Assoc. 111, 92-102.

Bruce, M.L., 2010. Suicide risk and prevention in veteran populations. Ann. N.Y. Acad. Sci. 1208, 98-103.

Daniels, K., Loganathan, M., Wilson, R., Kasckow, J.W., 2014. Appointment adherence in patients with schizophrenia. Clin. Care 11, 467-482.

Folstein, M.F., Folstein, S.E., McHugh, P.R., 1975. Mini-mental state. J. Psychiatr. Res. 12, 189-198.

Hamilton, M., 1960. A rating scale for depression. J. Neurol. Neurosurg. Psychiatry 23, 56-62.

Heneghan, C., Alonso-Coello, P., Garcia-Alamino, J., Perera, R., Meats, E., Glasziou, P., 2006. Self-monitoring of oral anticoagulation: a systematic review and metaanalysis. Lancet 367, 404-411.

Hoffmire, C.A., Kemp, J.E., Bossarte, R.M., 2015. Changes in suicide mortality for veterans and Nonveterans by gender and history of VHA service use, 20002010. Psychiatric Services. 2015 May 1:appips201400031 2015. (Epub ahead of print).

Kasckow, J., Felmet, K., Appelt, C., Thompson, R., Rotondi, A., Haas, G., 2014. Telepsychiatry in the assessment and treatment of schizophrenia. Clin. Schizophr. Relat. Psychoses 8, 21-27A.

Kasckow, J., Felmet, K., Zisook, S., 2011. Managing suicide risk in patients with schizophrenia. CNS Drugs 25, 129-143.

Kasckow, J., Gao, S., Hanusa, B., Rotondi, A., Chinman, M., Zickmund, S., Gurklis, J., Fox, L., Cornelius, J., Richmond, I., Haas, G.L., 2015. Telehealth monitoring of patients with schizophrenia and suicidal ideation. Suicide Life Threat Behav., 17. http://dx.doi.org/10.1111/sltb.12154 (Epub ahead of print).

Kroenke, K., Spitzer, R.L., Williams, J.B., 2001. The PHQ-9: validity of a brief depression measure. J. Gen. Intern. Med. 16, 606-613.

Landis, J., Koch, G., 1977. The measurement of observer agreement for categorical data. Biometrics 33, 159-174.

Miller, W., Crabtree, B.F., 1992. Primary care research: a multi typology and qualitative road map. In: Crabtree, B.F., Miller, W.L. (Eds.), Doing Qualitative Research. Sage Press, London.

Reynolds 3rd, C.F., Degenholtz, H., Parker, L.S., Schulberg, H.C., Mulsant, B.H., Post, E., Rollman, B., 2001. Treatment as usual (TAU) control practices in the PROSPECT study: managing the interaction and tension between research design and ethics. Int. J. Geriatr. Psychiatry 16, 602-608.

Valenstein, M., Eisenberg, D., McCarthy, J.F., Austin, K.L., Ganoczy, D., Kim, H.M., Ganoczy, D., Kim, H.M., Zivin, K., Piette, J.D., Olfson, M., Blow, F.C., 2009. Service implications of providing intensive monitoring during High-risk periods for suicide among Va patients with depression. Psychiatr. Serv. 60, 439-444.

While, D., Bickley, H., Roscoe, A., Windfuhr, K., Rahman, S., Shaw, J., Appleby, L., Kapur, N., 2012. Implementation of mental health service recommendations in England and Wales and suicide before-and-after observational study. Lancet 379, 1005-1012.

SCHRES-08012; No of Pages 8

Schizophrenia Research xxx (2018) xxx-xxx



Contents lists available at ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres

A telemedicine platform to improve clinical parameters in paranoid schizophrenia patients: Results of a one-year randomized study

Marek Krzystanek a,13, Mariusz Borkowskia, Katarzyna Skaiacka b, Krzysztof Krysta a

a Department of Psychiatric Rehabilitation, Department of Psychiatry and Psychotherapy, Medical School of Silesia, Ziolowa 45/47 Street, 40-635 Katowice, Poland. b Institute of Psychology, University ofOpole, Kopernika 11A Street, 45-040 Opole, Poland

ARTICLE INFO


ABSTRACT


Article history:

Received 20 April 2018

Received in revised form 16 July 2018

Accepted 12 August 2018

Available online xxxx


Objective: The study objective was to test a smartphone-based MONEO platform designed to improve the clinical condition of paranoid schizophrenia patients. Telemedicine treatment is considered to be as effective as traditional treatment in outpatient clinics.

Keywords:

Paranoid schizophrenia

Telemedicine

Telephone-based intervention

Smartphone platform


Method: A total of 290 patients with paranoid schizophrenia in the symptomatic remission state were recruited to this 12-month multicenter, open-label randomized trial. A study group (n = 191) received a smartphone with the MONEO platform installed. Patients conducted cognitive training twice a week. Patients' mental state was assessed every month via teleconference. A placebo group (n = 99) received a platform with functionality limited to monthly teleconsultation and performing cognitive training every 6 months. The clinical status was measured using the Positive and Negative Syndrome Scale (PANSS), Calgary, and Clinical Global Impression-Severity (CGI-S) clinical scales.

Results: After 12 months, a significant reduction of symptoms was observed in the study group, as assessed using the Calgary (36%, P < 0.01) and PANSS (8.6%, P < 0.05) scales. Symptom reduction of 23.6% was also observed in the placebo group (P < 0.05, Calgary scale). In the study group, depression, positive symptoms, excitement, general psychopathology and disorganization subscales decreased significantly, while in the placebo group, only the depression subscale decreased. The greatest improvement of 11.2% (study group, P < 0.05), vs 16.2% (placebo group, P < 0.05), was observed for the depression subscale.

Conclusion: The MONEO platform was demonstrated to positively influence the clinical condition of individuals with paranoid schizophrenia. A lack of negative consequences associated with usage of the device was also reported.

© 2018 Published by Elsevier B.V.

1. Introduction

Schizophrenia is a chronic mental disorder which affects approximately 1% of the general population (Perala et al., 2007). Individuals with schizophrenia have a 2.6-fold increased risk of death, mainly due to suicide and cardiovascular disease (McGrath et al., 2008). Schizophrenia impairs several aspects of everyday life such as interpersonal relationships or professional activity, often limiting patient independence (Harvey, 2014). Treatment involves both pharmacologic and psychosocial interventions, however their efficacy is only moderate (Jaaskelainen et al., 2013). Poor medication adherence (including treatment discontinuation) (Kane et al., 2013) and limited access to psychosocial care (Tandon et al., 2010) can contribute significantly to a low recovery rate and high risk of relapse.

Considering the rapid progress in telecommunication technology in recent years, telemedicine has become a promising approach to improve the management of schizophrenia (Kasckow et al., 2014). Telepsychiatry can provide patients with convenient and immediate home-based access to medical consultations. This can be applied to the management of patients with schizophrenia, enabling better control of treatment adherence and increasing chances to receive proper psychosocial intervention. Both telephone-based platforms (Stentzel et al., 2015) and electronic medication dispensers (Frangou et al., 2005) have been proposed to be used to improve treatment compliance of patients with schizophrenia. According to the results of a 6-month evaluation of schizophrenic patients, a smartphone-based system significantly enhanced medication compliance in individuals with low treatment adherence (Krzystanek et al., 2015). However, in a 12month observation, no such improvement was achieved (Krzystanek et al., 2017). Telephone-based psychotherapy on the other hand, was proven to be effective in treating anxiety disorder (Brenes et al.,

So far, the application of telehealth technology in schizophrenia has focused on three modalities: telephone- and Internet-based

https://doi.org/10.1016/j.schres.2018.08.016

0920-9964/© 2018 Published by Elsevier B.V.

M. Krzystanek et al. / Schizophrenia Research xxx (2018) xxx-xxx

interventions and videoconferencing (Kasckow et al., 2014). Among telephone-based interventions, the availability of smartphone-based applications dedicated to individuals with schizophrenia increases (Firth and Torous, 2015). The results of the above-mentioned studies indicate that telemedicine, and in particular telephone- or smartphonebased interventions, might act as a promising and affordable tool for the care of patients with schizophrenia. A study of Ben-Zeev et al., who compared two recovery-oriented self-management techniques: a smartphone-based intervention (FOCUS) and a widely-used clinicbased intervention (Wellness Recovery Action Plan), showed that, as well as similar improvements in clinical outcomes observed for both technologies, mHealth technology was beneficial to standard therapy in terms of patient engagement (Ben-Zeev et al., 2018). Telemedicine was shown to improve both medication adherence and clinical status, as well as to decrease rates of hospitalization and emergency visits (Frangou et al., 2005; Spaniel et al., 2008). The telemonitoring platform @HOME employing electronic medicine dispensers was shown to significantly increase medical adherence of patients with schizophrenia from 75.3% observed for the control group to as much as 92.3% for telemedicine platform users (Frangou et al., 2005). At the same time, patients using the @HOME platform had greater improvements in clinical outcome (measured with the PANSS and CGI scales) than the control groups at the end of the study (Frangou et al., 2005). A significant decrease (60%) in the number of hospitalizations was achieved with the ITAREPS mobile phone platform for weekly patient monitoring, providing evidence that telemedicine might be beneficial in preventing psychotic relapse (Spaniel et al., 2008). The telehealth monitoring system reduced the hospitalization rate in veterans with schizophrenia from 32% to 5% when added to intensive case monitoring program (Flaherty et al., 2017). However, given the small sample size of the aforementioned analyses, there remains a need for further testing.

The possibility to collect and analyze a large amount of data regarding the patient's daily routine in the real-time triggered the development of smartphone applications oriented toward early identification of psychotic relapse. A smartphone-based data collection system, CrossCheck, captures data regarding the user's behavior, such as physical activity or speech frequency, and smartphone usage (Ben-Zeev et al.,

Videoconferencing has proven to be well-accepted and tolerated among patients with schizophrenia, and the clinical assessment obtained was shown to be equivalent to an in-person examination (Sharp et al., 2011). This demonstrates the possibility of remotely conducting reliable psychiatric assessment of psychotic patients, further increasing the feasibility of telepsychiatry not only to improve treatment efficacy, but also to provide an early diagnosis and remotely manage the condition of psychotic patients.

In this study, we described the potential of a new smartphone-based MONEO platform to improve the clinical condition of paranoid schizophrenia patients. As secondary objectives, we investigated whether use of the MONEO platform influenced the stability of the patients' clinical condition, rates of hospitalization, and visits to an outpatient clinic.

2. Methods

2.1. Study design, participants, and randomization

The study was a multicenter, open-label, randomized trial. A total of 290 patients with paranoid schizophrenia were enrolled from 27 centers in Poland; 199 constituted the study group, and 91, the placebo group. All patients were Caucasian, aged between 18 and 45 years, with paranoid schizophrenia diagnosed within the past 10 years. All enrolled patients were in the state of symptomatic remission (the severity of symptoms was not greater than mild and did not affect daily functioning and behavior) and their schizophrenic symptoms were stable at a mild level (enabling daily functioning) for at least 6 months prior to study enrollment. All study participants confirmed constant access to a high-speed Internet connection (3G). Exclusion criteria were as follows: a co-existing psychiatric condition (particularly schizophreniclike syndromes or organic psychotic disorders); an unstable mental (acute episodes in the past 6 months) or physical state (serious or chronic somatic disease); participation in another clinical trial in the past 6 months; lack of ability to use an electronic device with a touch screen; pregnancy or lactation; any other reason that, according to the investigator, prevented the individual from participating in a clinical study.

Randomization was performed by the MONEO system after patient registration. Patients were randomized either to the study group (who received a full version of the MONEO application) or to the placebo group (who received an inactive version of the application with limited functionality). All enrolled patients gave written informed consent to participate in the study. Recruitment took place from January to July 2014. Follow-up ended in November 2015. The study was registered by the Polish Office for Registration of Medicinal Products, Medical Devices, and Biocides under the ID number UR.DNB.4501.0024.2013.

2.2. Intervention and implementation

After enrollment, each patient received a smartphone with the MONEO telemedicine platform installed (for detailed description, see Supplementary Data).

The software in the study group reminded the patient of the need to take medication 1 h before and after the scheduled time. Medical adherence was monitored based on feedback messages sent by the patient. The software enabled the patient to conduct cognitive training twice a week. The patient received the reminder about scheduled training one day before the training and completed it of his/her own will. Software also contained a library of videos and audiobooks accessible to the patients at will. Patients were able to report the need for a televisit, with the investigator providing an answer immediately, or, if not available, as soon as possible. The investigator assessed the mental state of the patient using psychometric scales every month during a 12-month study via teleconference. Visits to an outpatient clinic were arranged at least once every 3 months.

The placebo group was provided with an inactive version of the software, which only allowed for: a monthly examination by an investigator (using a clinical scale, as previously defined) during a videoconference; performance of cognitive training at the beginning of the trial, then after 6 and 12 months of the trial; and registration for an outpatient visit.

During the study, patients were treated with any necessary psychiatric medications to maintain their health. The study made no changes to current pharmacological treatment unless the investigator deemed this necessary during a televisit. All adverse events associated with the use of the platform or medications taken were noted by the investigator.

2.3. Outcomes and measures

The level of improvement in the clinical status of patients associated with the use of the MONEO platform was measured by the percentage change in the total score obtained from the clinical assessment scales (see below) after 6 and 12 months, compared with that at the beginning of the study. The difference in the mean severity level of clinical symptoms was measured by the global score in the clinical scales listed below at the beginning and after 6 and 12 months of the study, within and between the study and placebo groups.

The assessment of the stability of the clinical status of patients in the study and placebo groups was expressed as the time elapsed to deterioration in clinical scale scores by at least 20%, compared with values recorded at the beginning of the study, within and between groups.

M. Krzystanek etal./ Schizophrenia Research xxx (2018) xxx-xxx

The following clinical scales were used to assess the clinical condition of the patients:

Analysis of clinical symptom dimensions of the PANSS scale, conducted both within and between the study and placebo groups, was based on the total number of points in each of the following subscales of the PANSS scale: positive symptoms (sum of P1-P7 items); negative symptoms (sum ofN1-N7 items); symptoms of hostility/excitement (sum of P4, P7, O4, O8, and O14 items); symptoms of anxiety/depres-sion (sum of O1-O3, O6, and O15 items); symptoms of disorganization (sum of P2, N5, O5, O10, O11, O13, and O15 items); and cognitive symptoms (sum of N5, O10, P2, P6, and N7 items).

The number of hospitalizations and visits to outpatient clinics was gathered directly from the MONEO platform. The total number of such incidents after 12 months of the trial was compared between the study and placebo groups.

The safety of the study protocol was assessed by the incidence of adverse effects associated with the use of the MONEO platform and was compared both within and between the study and placebo groups. The occurrence of adverse effects was determined by the investigators during televisits with the patients.

2.4. Statistical analysis

Final analysis was performed for all patients who completed the scheduled procedures throughout the study. Missing data were omitted without imputation. All statistical analyses were conducted using STATISTICA10 (StatSoft) software. Continuous variables were described with the number of non-missing observations, arithmetic mean, standard deviation, median, quartiles, and range. Categorical variables were expressed with the number of non-missing observations and percentages. Univariate and multivariate statistical tests with repeated measurement for dependent groups were applied (Student's t-test, Wilcoxon test, ANOVA). For multivariate analysis measurements, post-hoc tests were applied (Tukey test, LSD test, Scheffe test, Games-Howell test). To counteract the problem of multiple comparisons, the Bonferroni correction was used (Dunn, 1961). All data were controlled for equality of variances and normal distribution. If these assumptions were not met, the Welch-type adjustments were used. To assess clinical stability, the number of hospitalizations and visits to outpatient clinics were compared between the study and placebo groups with McNemar's test and Cochran's Q ANOVA applied. A two-tailed P value < 0.05 was considered statistically significant.

3. Results

3.1. Baseline patient characteristics

Initially, 300 patients were enrolled in the study. Due to lack of signed informed consent (n = 9) or signed consent for personal data processing (n = 1), 10 patients were excluded, giving a total n = 290 (study group = 199 and placebo group = 91). Fig. 1 shows the flow of patients throughout the study, and Table 1 shows baseline demographic and clinical characteristics of enrolled patients. All participants were Caucasian, the majority (60%) were male and the mean age was

3.2. Clinical outcomes

Changes in the clinical status of the patients within the study and placebo groups are listed in Table 2. After 12 months of the study, patients using the active version of the MONEO platform reported a significant reduction in both affective symptoms (by 36%, P < 0.01) and in the symptoms assessed with the PANSS scale (by 8.6%, P < 0.05), whereas the improvement of the clinical status (CGI-S scale) was not significant (3.4%, P > 0.05). Within the placebo group, the decrease in the symptoms was assessed as significant only when using the Calgary scale (23.6%, P < 0.05), and insignificant using the PANSS (10.8%, P > 0.05) and CGI-S (8.5%, P > 0.05) scales. The interactional effects between the study and placebo groups were significant only for the PANSS scale (P < 0.05).

To assess the stability of the patients, we compared their clinical symptoms measured every month throughout the study. Changes in clinical symptoms during the study (in both study and placebo groups) did not reach a critical 20% level on the CGI-S and PANSS scales, indicating the stability of the clinical condition of patients. Considering the study group, the initial results of the PANSS total score were significantly different from those obtained after the fifth, eleventh, and twelfth month of the study (P < 0.05). No significant differences between the study and the placebo group within all subsequent study periods were observed.

3.3. Clinical symptom dimensions

To further investigate the changes in the clinical status of patients during the study, we assessed the dimensions of the clinical symptoms given on the subscales of the PANSS score (Table 3). In the study group, the decrease in the symptoms was significant for positive symptoms, general psychopathology, excitement, depression, and disorganization subscales, while insignificant for negative and cognitive symptoms subscales. In the placebo group, only symptoms on the depression subscale declined significantly. The greatest improvement was observed in the depression subscale: symptoms of anxiety and depression were reduced by 11.2% after 12 months in the study group (P < 0.05) and by 16.2% in the placebo group (P < 0.05).

3.4. Frequency of hospitalizations and visits to outpatient clinics

Both the number of hospitalized patients and the length of hospitalization were similar between the study and placebo groups (P > 0.05). However, the number of analyzed cases was low (n = 20 and 8, for the study and placebo groups, respectively). No cases of hospitalization were related to the use of the MONEO platform. Similarly, the number of visits to outpatient clinics did not differ between the study and placebo groups (mean number of visits = 7.8 and 6.8 for the study and placebo groups, respectively [P> 0.05]).

3.5. Adherence to MONEO platform usage

The adherence to MONEO platform usage was defined as the number of patients who underwent visits by video call (the physician calls the patient) or as the number of the patients' visits to the outpatient clinic at the particular timepoints of the study. According to the schedule, the patients should undergo one obligatory video call visit every month and one obligatory visit to the outpatient clinic every three months. The data regarding patients' adherence for both study and placebo groups is shown in Table 4.

3.6. Safety data

During the study, a total of 63 adverse events were reported, of which 12 (19%) were related to pharmacotherapy. These events


occurred in 34 patients (11.7% of all patients). The study and placebo groups did not differ significantly with regard to the seriousness of the reported adverse cases or relationship of the cases to the smartphone device (P > 0.05). A total of seven cases were related to the device, of which four (2%) were observed in the study group and three (3.3%) in the placebo group (Table 5). The majority of device-related cases (5 of 7) were due to device failure. All adverse events related to the device are listed ir Table 6. Reported adverse events coded in System Organ Class and in Preferred Term are listed in Table S1.

4. Discussion

The rapid increase in the availability of mobile phones, together with the growing popularity of telemedical services, presents an opportunity to develop a new, promising approach to treat schizophrenia patients using smartphone-based platforms. Regular monitoring of patients using a smartphone device might be beneficial not only for preventing relapse, but such monitoring allows for the possibility of conducting remote interventions which can improve patients' clinical outcomes, particularly when face-to-face access to a psychiatrist is restricted.

M. Krzystanek etal./ Schizophrenia Research xxx (2018) xxx-xxx

Table 1

Baseline demographic and clinical characteristics of the patients.

Characteristic

Study group

(n = 199)

Placebo group

(n = 91)

P

Mean age (years)

32.0 (5.92)

32.2 (6.94)

0.8

Sex - male

114 (57.3%)

60 (65.9%)

0.16

Race - Caucasian

199 (100%)

91 (100%)

1

Clinical status

Total PANSS score

58.0 (20.3)

59.8 (23.7)

0.51

Calgary scale

4.0 (4.2)

3.4 (4.1)

0.26

CGI-S scale

2.7 (1.0)

2.7 (1.1)

1.0

PANSS subscales

PANSS positive symptoms

12.8 (5.4)

12.3 (5.3)

0.46

PANSS negative symptoms

16.4 (7.8)

16.5 (6.6)

0.92

PANSS general psychopathology

30.6 (12.4)

30.2 (10.6)

0.79

PANSS excitement

8.8 (3.5)

8.9 (3.6)

0.82

PANSS depression

9.6 (4.2)

9.5 (3.9)

0.85

PANSS disorganization

14.0 (6.1)

13.2 (4.9)

0.27

PANSS cognitive symptoms

10.4 (4.7)

9.8 (3.9)

0.29

Data in parentheses show the percentage of patients, unless otherwise indicated. Age and clinical scale scores are presented as mean values with the standard deviation shown in parenthesis, alpha = 0.05.

The use of digital devices among 457 individuals with schizophrenia was recently assessed with a web-based survey by Gay et al. (2016). A great majority of respondents (411/457, 90%) declared access to at least two digital items such as computers, mobile phones, landline phones, or tablets. The devices were frequently used: as much as 89% of survey participants spent 1 h or more each day on a personal computer, and 85% on mobile phones. Respondents used the technology to deal with schizophrenia in several ways: to manage auditory hallucinations with audio files (42%), to seek information about mental health (38%), to be reminded about appointments (37%), to manage their treatment schedule (28%), to develop relationships with others living with schizophrenia (26%), to monitor symptoms (25%), and to identify coping strategies (24%). The MONEO platform, described in this study, addresses most of the aforementioned needs. In particular, it enables one to explore information about mental health and find coping strategies by providing access to the library of audiobooks and videos, it reminds patients about appointments and the necessity of taking medications, and it helps to control symptoms in a regular manner by facilitating medical televisits. Two-thirds of the respondents declared that digital technology will play a central role in their recovery in the near future, underlining the demand for the development of telemedical tools dedicated to the management of schizophrenia (Gay et al., 2016).

Similarly to the general population, the ownership of mobile phones among people with schizophrenia is constantly increasing, and exceeded 80% in 2015 (Firth et al., 2016). More importantly, a large study of 1592 patients with severe mental illnesses revealed that 81% of those who owned a mobile phone expressed an interest in receiving mHealth support (Ben-Zeev et al., 2013). Smartphone technologies can provide convenient tools for self-monitoring, training, and facilitating patient's self-management of the disease (Luxton et al., 2011). Although availability of smartphones increases and their application in improving outcomes of patients with schizophrenia is promising, their implementation in clinical practice is still poorly supported by empirical evidence (Firth and Torous, 2015). The use of telemedicine devices (in particular videoconferences) in psychiatry has raised serious concerns, including a possible low acceptance rate of such virtual contact, among psychotic patients. However, recent studies showed that videoconferences are not only well-tolerated and accepted by patients (without deterioration of their psychotic symptoms) (Sharp et al., 2011), but also, in some cases, can be more beneficial than in-person consultations. The increased distance between patient and medical professional obtained via a videoconference was demonstrated to be less anxiety-provoking and minimized the risk of overstimulation (Magaletta et al., 2000).

Currently, research into telehealth technologies in patients with schizophrenia has focused mostly on interventions based on telephones, particularly smartphones, whereas studies considering Internet- and video-based approaches have largely been centered around the development of these platforms and the assessment of their feasibility, rather than the clinical management of patients themselves (Kasckow et al., 2014). The MONEO platform described in this study combines the advantages of all three modalities. Being a smartphone-based system, it enables one to hold videoconferences to remotely assess patient condition, and, similarly to Internet-based systems, it allows the patient to conduct cognitive training and self-educate using a library of available audiobooks and videos.

A wide range of possibilities offered by smartphones are being successfully utilized for management of schizophrenia. The ClinTouch application, described in 2012, serves as a platform to obtain selfassessment of psychiatric symptoms by PANSS and Calgary scales up to six times a day. The results delivered with the application were clinically meaningful and correlated with those obtained during face-to-face interviews (Palmier-Claus et al., 2012). The

Table 2

Change in the overall clinical status for all clinical scales considered (study and placebo groups).

Group

Clinical scale

Time

N

Mean (SD)

Difference (T0-T12)

Median (IQR)

Min-max

P (T0-T12)

Study

CGI-S

T0

165

2.7 (1.0)

-3.4%

3.0 (1.0)

1.0-5.0

0.38

T6

166

2.5 (1.0)

2.0 (1.0)

1.0-6.0

T12

121

2.6 (0.9)

3.0 (1.0)

1.0-5.0

CALGARY

T0

174

4.0 (4.2)

-36.1%

3.0 (6.0)

0.0-20.0

0.003

T6

170

2.6 (3.6)

1.0 (4.0)

0.0-20.0

T12

122

2.6 (3.4)

1.0 (4.0)

0.0-13.0

PANSS

T0

168

58.0 (20.3)

-8.6%

53.0 (32.5)

30.0-115.0

0.035

T6

169

54.9 (20.9)

50.0 (32.0)

30.0-113.0

T12

122

53.0 (19.3)

49.0 (26.0)

30.0-122.0

Placebo

CGI-S

T0

76

2.7 (1.1)

-8.5%

3.0 (1.0)

1.0-5.0

0.332

T6

64

2.7 (1.2)

3.0 (2.0)

1.0-5.0

T12

46

2.5 (1.0)

2.0 (1.0)

1.0-5.0

CALGARY

T0

81

3.4 (4.1)

-23.6%

2.0 (5.0)

0.0-18.0

0.048

T6

66

2.6 (3.2)

1.0 (4.0)

0.0-12.0

T12

46

2.6 (4.1)

0.0 (4.0)

0.0-14.0

PANSS

T0

79

59.8 (23.7)

-10.8%

55.0 (33.0)

30.0-129.0

0.123

T6

65

57.5 (21.9)

54.0 (39.0)

30.0-106.0

T12

46

53.3 (20.6)

48.5 (36.0)

30.0-107.0

N, number of observations; SD, standard deviation; IQR, interquartile range; T0, baseline; T6, after 6 months; T12, end of the study (after 12 months). Significant P values between T0-T12 are marked in bold, alpha = 0.05.

M. Krzystanek et al. / Schizophrenia Research xxx (2018) xxx-xxx

Table 3

Analysis of clinical symptoms using the PANSS subscales (study and placebo groups).

GROUP

PANSS subscale

Time

N

Mean

(SD)

Difference (T0-T12)

Median (IQR)

Min-max

P (T0-T12)

Study

Positive symptoms

T0

170

12.2 (5.2)

-8.8%

11.0 (7.0)

7.0-28.0

0.043

T12

122

11.1 (5.2)

9.0 (5.5)

7.0-30.0

Negative symptoms

T0

170

16.2 (6.5)

-7.6%

15.0 (11.0)

7.0-32.0

0.0102

T12

122

15.0 (5.7)

14.0 (9.0)

7.0-30.0

General psychopathology

T0

168

29.6 (10.4)

-9.0%

27.0 (17.0)

16.0-60.0

0.035

T12

122

27.0 (10.2)

24.0 (14.0)

16.0-62.0

Excitement

T0

168

8.7 (3.5)

-9.9%

8.0 (5.0)

5.0-19.0

0.0308

T12

122

7.8 (3.3)

7.0 (4.0)

5.0-19.0

Depression

T0

168

9.4 (3.8)

-11.2%

9.0 (6.0)

5.0-21.0

0.0124

T12

122

8.3 (3.5)

7.0 (6.0)

5.0-18.0

Disorganization

T0

168

13.0 (4.8)

-7.3%

12.0 (7.5)

7.0-30.0

0.0471

T12

122

12.1 (4.9)

11.0 (7.0)

7.0-31.0

Cognitive symptoms

T0

168

9.7 (3.8)

-7.1%

8.5 (6.0)

5.0-21.0

0.118

T12

122

9.0 (3.7)

8.0 (5.0)

5.0-23.0

Placebo

Positive symptoms

T0

79

12.8 (5.4)

-13.4%

11.0 (8.0)

7.0-27.0

0.069

T12

46

11.1 (4.2)

9.5 (7.0)

7.0-21.0

Negative symptoms

T0

79

16.4 (7.8)

-10.2%

16.0 (13.0)

7.0-35.0

0.247

T12

46

14.8 (6.7)

14.0 (9.0)

7.0-31.0

General psychopathology

T0

79

30.6 (12.4)

-10.0%

28.0 (16.0)

16.0-71.0

0.166

T12

46

27.5 (11.3)

26.0 (16.0)

16.0-58.0

Excitement

T0

79

8.8 (3.5)

-7.0%

8.0 (5.0)

5.0-18.0

0.352

T12

46

8.2 (3.4)

7.0 (6.0)

5.0-18.0

Depression

T0

79

9.6 (4.2)

-16.2%

8.0 (6.0)

5.0-24.0

0.033

T12

46

8.0 (3.6)

6.5 (6.0)

5.0-18.0

Disorganization

T0

79

14.0 (6.1)

-11.1%

12.0 (10.0)

7.0-30.0

0.169

T12

46

12.5 (5.4)

11.0 (10.0)

7.0-25.0

Cognitive symptoms

T0

79

10.4 (4.7)

-12.8%

9.0 (7.0)

5.0-22.0

0.116

T12

46

9.1 (3.9)

8.5 (7.0)

5.0-19.0

N, number of observations; SD, standard deviation; IQR, interquartile range; T0, baseline; T6, after 6 months; T12, end of the study (after 12 months). Significant P values between T0-T12 are marked in bold, alpha = 0.05.

feasibility of smartphone-based applications for individuals with schizophrenia, which were superior to those of conventional mobile phones, was demonstrated by Ainsworth et al.; participants completed application-based surveys faster and more often than surveys provided by text-message platform (Ainsworth et al., 2013). The FOCUS application, on the other hand, offers several tools that support the self-management of schizophrenia addressing medication adherence, social functioning, mood problems, auditory hallucinations, and sleep difficulties (Ben-Zeev et al., 2014). The application offers an individualized intervention based on results of the survey completed by the patient. Naslund et al. provided evidence that a smartphone-based, physical activity-tracking Fitbit application, designed for the general population, is both feasible and well-accepted by individuals with serious mental disorders, including schizophrenia (Naslund et al., 2015). Another smartphone application, WellWave, aims to promote physical well-being in adults with psychiatric disorders by encouraging mild physical exercise and offering the possibility of confidential text-messaging with program staff and access to a library of motivational ebooks and videos. It was proven to engage participants in physical activity and other activities that promote a healthy lifestyle, with a 73% mean response rate across all sent messages (Macias et al., 2015). Recently the Actissist application was proven to be feasible, acceptable, and safe in a randomized, controlled trial with early psychosis patients (Bucci et al., 2018). The application enables the patient to complete

Table 4

Adherence to telemedical treatment with the MONEO platform throughout the study assessed as a number of completed video call visits.

Timepoint

Study group N = 199

Placebo group

N = 91

T0

170 (85%)

79 (87%)

T6

169 (85%)

65 (71%)

T12

122 (61%)

46 (51%)

T0, baseline; T6, after 6 months; T12, end of the study (after 12 months).

a self-assessment set of questions, and, based on its results, it provides the patient with a range of activities such as mindfulness exercises, educational videos, fact sheets, and external links, designed to alleviate psychotic symptoms, as well as emergency contact resources. The graphical summary of weekly activities supports the patient in self-assessment of symptoms and may contribute to decision-making regarding treatment (Bucci et al., 2018).

The adherence to smartphone applications among patients with schizophrenia and other psychiatric disorders varied between 69% and 94% of all days or possible entries (Firth and Torous, 2015). This percentage, although obtained for short-term observations, is in agreement with the adherence data gathered for the 12-month study on the MONEO platform, which ranged from 61% to 85%.

In this study, we demonstrated that the clinical condition of 199 patients with schizophrenia using a smartphone-based MONEO platform was stable throughout a 1 -year trial. Moreover, we showed that patients using the full version of the MONEO platform exhibited a pronounced reduction in the schizophrenic symptoms assessed using the Calgary

Table 5

Characteristics of adverse events (excluding those related to pharmacotherapy) reported during the 12 months of the study.

Study group

(N = 199)

Placebo group

(N = 91)

Overall (N = 290)

Seriousness

Not serious

9 (4.5%)

3 (3.3%)

12 (4.1%)

Serious

33 (16.6%)

6 (6.6%)

39 (13.4%)

Relatedness to device

Related

4 (2.0%)

3 (3.3%)

7 (2.4%)

Not related

39 (19.6%)

6 (6.6%)

45 (15.5%)

Subject outcome

Recovered/resolved

38 (19.1%)

4 (4.4%)

42 (14.5%)

Not recovered/resolved

1 (0.5%)

0 (0%)

1 (0.3%)

On-going

0 (0%)

3 (3.3%)

3 (1.0%)

Death

1 (0.5%)

0 (0%)

1 (0.3%)

Unknown

22 (1.0%)

2 (2.2%)

4 (1.4%)

M. Krzystanek etal./ Schizophrenia Research xxx (2018) xxx-xxx

Table 6

Description of all cases of adverse events related to the smartphone device tested, reported during the 12 months of the study.

System organ class

Preferred term

Study group (N = 199)

Placebo group (N = 91)

Overall (N = 290)

General disorders and administration site conditions

Devicefailure

1.5% (n = 3)

2.2% (n = 2)

1.7% (n = 5)

Psychiatric disorders

Irritability

0.0% (n = 0)

1.1% (n = 1)

0.3% (n = 1)

NS

NA

0.5% (n = 1)

0.0% (n = 0)

0.3% (n = 1)

Expectedness according to protocol

2.0% (n = 4)

3.3% (n = 3)

2.4% (n = 7)

N, number of observations; NS, not specified; NA, not applicable.

and PANSS scales, whereas such improvement was weaker and statistically significant only when assessed using the Calgary scale, considering those individuals supplied with an inactive version of the software (placebo group). Analysis of symptom dimensions on the PANSS scale revealed that the study group improved significantly in 5 out of 7 subscales, while the reduction of symptoms within the placebo group reached statistical significance only for the depression subscale. Both study and placebo groups consisted of patients in the state of symptomatic remission. Therefore, observed slight however statistically significant improvement is indicative of a clinically pronounced reduction of symptoms toward further normalization of a psychic state and complete functional recovery.

Use of the MONEO platform did not influence the rate of hospitalization and visits to outpatient clinics. This may be because the population under study was young and only displayed mild symptoms of the illness. Additionally, the stability of the patients demonstrated here in the study suggests the potential of the MONEO platform in preventing considerable deterioration of symptoms. The MONEO platform was safe, with the frequency of adverse events in the study group being similar to that observed in the placebo group, as well as that detailed in other reports (Frangou et al., 2005; Salzer et al., 2004; Spaniel et al., 2008).

One of the concerns regarding telemedical platforms is their possible implementation in middle-aged or elderly patients with limited technical abilities. This issue seems to be particularly relevant for individuals with serious mental illnesses, who may additionally experience reduced cognitive functions. Therefore, our study was conducted on a group of young patients aged between 18 and 45 years. However, Whiteman et al. provided evidence that a smartphone-based platform could be successfully adapted to meet the needs of older patients (Whiteman et al., 2017).

The present study does, however, have certain limitations. Several centers recruited only 1 patient, which could result in bias regarding patient care. No data regarding the patients' pharmacotherapy were provided, which might have been useful when assessing symptom improvement. One of the secondary endpoints could not be fully analyzed as patients refused to disclose data regarding their hospitalizations in the 12 months prior to the commencement of the study. Finally, although the improvement in clinical status was generally higher in the study vs the placebo group, only one significant difference between the groups was shown.

5. Conclusions

The MONEO platform was demonstrated to positively influence the clinical condition of individuals with paranoid schizophrenia. The results obtained demonstrated both the stability of the patients and the lack of negative consequences associated with use of the smartphone device. As improvement was achieved in both the study and placebo groups, it might be hypothesized that possession of the device, irrespective of its functionality, provided the patients with a feeling of safety, contributing to the reduction of their symptoms. Therefore, to further examine the potential of the MONEO platform, it would be pertinent to compare patients using the device with those who do not have access to it at all.

Funding source

The study was financed by The National Centre for Research and Development (grant number: POIG.01.04.00-04-219/12).

Contributions

Marek Krzystanek designed the study, wrote the protocol and wrote the first draft of the manuscript. Mariusz Borkowski and Krzysztof Krysta managed the literature searches and analyses. Katarzyna Skalacka and Marek Krzystanek undertook the statistical analysis. All authors contributed to and have approved the final manuscript.

Conflict of interest

All authors declare no conflicts of interest.

Acknowledgments

No acknowledgments.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.schres.2018.08.016.

References

Ainsworth, J., Palmier-Claus, J.E., Machin, M., Barrowclough, C., Dunn, G., Rogers, A., Buchan, I., Barkus, E., Kapur, S., Wykes, T., Hopkins, R.S., Lewis, S., 2013. A comparison of two delivery modalities of a mobile phone-based assessment for serious mental illness: native smartphone application vs text-messaging only implementations. J. Med. Internet Res. 15, e60.

Ben-Zeev, D., Davis, K.E., Kaiser, S., Krzsos, I., Drake, R.E., 2013. Mobile technologies among people with serious mental illness: opportunities for future services. Admin. Pol. Ment. Health 40, 340-343.

Ben-Zeev, D., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., Mueser, K.T., 2014. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr. Bull. 40, 1244-1253.

Ben-Zeev, D., Brian, R., Wang, R., Wang, W., Campbell, A.T., Aung, M.S.H., Merrill, M., Tseng, V.W.S., Choudhury, T., Hauser, M., Kane, J.M., Scherer, E.A., 2017. CrossCheck: integrating self-report, behavioral sensing, and smartphone use to identify digital indicators of psychotic relapse. Psychiatr. Rehabil. J. 40, 266-275.

Ben-Zeev, D., Brian, R.M., Jonathan, G., Razzano, L., Pashka, N., Carpenter-Song, E., Drake, R.E., Scherer, E.A., 2018. Mobile health (mHealth) versus clinic-based group intervention for people with serious mental illness: a randomized controlled trial. Psychiatr. Serv. https://doi.org/10.1176/appi.ps.201800063.

Brenes, G.A., Danhauer, S.C., Lyles, M.F., Hogan, P.E., Miller, M.E., 2015. Telephone-delivered cognitive behavioral therapy and telephone-delivered nondirective supportive therapy for rural older adults with generalized anxiety disorder: a randomized clinical trial. JAMA Psychiat. 72, 1012-1020.

Bucci, S., Barrowclough, C., Ainsworth, J., Machin, M., Morris, R., Berry, K., Emsley, R., Lewis, S., Edge, D., Buchan, I., Haddock, G., 2018. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr. Bull. https://doi.org/ 10.1093/schbul/sby032.

Dunn, O.J., 1961. Multiple comparisons among means. J. Am. Stat. Assoc. 56, 52-64.

Firth, J., Torous, J., 2015. Smartphone apps for schizophrenia: a systematic review. JMIR Mhealth Uhealth 6, e102.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2016. Mobile phone ownership and endorsement of "mHealth" among people with psychosis: a meta-analysis of cross-sectional studies. Schizophr. Bull. 42, 448-455.

Flaherty, L.R., Daniels, K., Luther, J., Haas, G.L., Kasckow, J., 2017. Reduction of medical hospitalizations in veterans with schizophrenia using home telehealth. Psychiatry Res. 255, 153-155.

Frangou, S., Sachpazidis, I., Stassinakis, A., Sakas, G., 2005. Telemonitoring of medication adherence in patients with schizophrenia. Telemed. J. E Health 11, 675-683.

Gay, K., Torous, J., Joseph, A., Pandya, A., Duckworth, K., 2016. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Ment. Health 3, e15.

Harvey, P.D., 2014. Assessing disability in schizophrenia. J. Clin. Psychiatry 75, e27.

Jaaskelainen, E., Juola, P., Hirvonen, N., McGrath, J.J., Saha, S., Isohanni, M., Veijola, J., Miettunen, J., 2013. A systematic review and meta-analysis of recovery in schizophrenia. Schizophr. Bull. 39, 1296-1306.

8                                                          M. Krzystanek et al. / Schizophrenia Research xxx (2018) xxx-xxx

Kane, J.M., Kishimoto, T., Correll, C.U., 2013. Non-adherence to medication in patients with psychotic disorders: epidemiology, contributing factors and management strategies. World Psychiatry 12, 216-226.

Kasckow,J., Felmet, K., Appelt, C., Thompson, R., Rotondi, A., Haas, G., 2014. Telepsychiatry in the assessment and treatment of schizophrenia. Clin. Schizophr. Relat. Psychoses 8, 21-27A.

Krzystanek, M., Krzeszowski, D.,Jagoda, K., Krysta, K., 2015. Long term telemedicine study of compliance in paranoid schizophrenia. Psychiatr. Danub. 27 (Suppl. 1), S266-S268.

Krzystanek, M., Krysta, K., Skalacka, K., 2017. Treatment compliance in the long-term paranoid schizophrenia telemedicine study. J. Technol. Behav. Sci. 2 (2), 84-87. https://doi.org/10.1007/s41347-017-0016-4 Epub 2017 May 18.

Luxton, D.D., McCann, R.A., Bush, N.E., Mishkind, M.C., Reger, G.M., 2011. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof. Psychol. Res. Pract. 42, 505-512.

Macias, C., Panch, T., Hicks, Y.M., Scolnick, J.S., Weene, D.L., Ongur, D., Cohen, B.M., 2015. Using smartphone apps to promote psychiatric and physical well-being. Psychiatry Q. 86, 505-519.

Magaletta, P.R., Fagan, T.J., Peyrot, M.F., 2000. Telehealth in the Federal Bureau of Prisons: inmates' perceptions. Prof. Psychol. Res. Pract. 31,497-502.

McGrath,J., Saha, S., Chant, D., Welham,J., 2008. Schizophrenia: a concise overview of incidence, prevalence, and mortality. Epidemiol. Rev. 30,67-76.

Morland, L.A., Mackintosh, M.-A., Greene, C.J., Rosen, C.S., Chard, K.M., Resick, P., Frueh, B.C., 2014. Cognitive processing therapy for posttraumatic stress disorder delivered to rural veterans via telemental health: a randomized noninferiority clinical trial. J. Clin. Psychiatry 75, 470-476.

Naslund, J.A., Aschbrenner, K.A., Barre, L.K., Bartels, S.J., 2015. Feasibility of popular m-health technologies for activity tracking among individuals with serious mental illness. Telemed. J. E Health 21, 213-216.

O'Neil, A., Taylor, B., Hare, D.L., Sanderson, K., Cyril, S., Venugopal, K., Chan, B., Atherton,J.J., Hawkes, A., Walters, D.L., Oldenburg, B., MoodCare InvestigatorTeam, 2015. Longterm efficacy of a tele-health intervention for acute coronary syndrome patients with depression: 12-month results of the MoodCare randomized controlled trial. Eur. J. Prev. Cardiol. 22, 1111-1120.

Palmier-Claus, J.E., Ainsworth, J., Machin, M., Barrowclough, C., Dunn, G., Barkus, E., Rogers, A., Wykes, T., Kapur, S., Buchan, I., Salter, E., Lewis, S.W., 2012. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry 12, 172.

Perala,J., Suvisaari, J., Saarni, S.I., Kuoppasalmi, K., Isometsa, E., Pirkola, S., Partonen, T., Tuulio-Henriksson, A., Hintikka, J., Kieseppa, T., Harkanen, T., Koskinen, S., Lonnqvist, J., 2007. Lifetime prevalence of psychotic and bipolar I disorders in a general population. Arch. Gen. Psychiatry 64, 19.

Preschl, B., Maercker, A., Wagner, B., 2011. The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression. BMC Psychiatry 11, 189.

Salzer, M.S., Tunner, T., Charney, N.J., Kopke, W., Muller, P., Muller-Spahn, F., Pietzcker, A., Tegeler, J., 2004. A low-cost, telephone intervention to enhance schizophrenia treatment: a demonstration study. Schizophr. Res. 66, 75-76.

Sharp, I.R., Kobak, K.A., Osman, D.A., 2011. The use of videoconferencing with patients with psychosis: a review of the literature. Ann. General Psychiatry 10, 14.

Spaniel, F., Vohlidka, P., Hrdlicka,J., Kozeny,J., Novak, T., Motlova, L., Cermak,J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res. 98, 312-317.

Stentzel, U., Grabe, H.-J., Strobel, L., Penndorf, P., Langosch, J., Freyberger, H.J., Hoffmann, W., van den Berg, N., 2015. Tecla: a telephone- and text-message based telemedical concept for patients with severe mental health disorders-study protocol for a controlled, randomized, study. BMC Psychiatry 15, 273.

Tandon, R., Nasrallah, HA, Keshavan, M.S., 2010. Schizophrenia, just the facts5. Treatment and prevention past, present, and future. Schizophr. Res. 122, 1-23.

Whiteman, K.L., Lohman, M.C., Gill, L.E., Bruce, M.L., Bartels, S.J., 2017. Adapting a psychosocial intervention for smartphone delivery to middle-aged and older adults with serious mental illness. Am. J. Geriatr. Psychiatry 25, 819-828.

Original Paper

Smartphone-Enhanced Symptom Management In Psychosis: Open, Randomized Controlled Trial

Shon Lewis1, MD; John Ainsworth1, PhD; Caroline Sanders1, PhD; Charlotte Stockton-Powdrell1, MA; Matthew Machin1, BEng; Pauline Whelan1, PhD; Richard Hopkins1, PhD; Zhimin He2, PhD; Eve Applegate1, PhD; Richard Drake1, PhD; Charlie Bamford1, PhD; Chris Roberts1, PhD; Til Wykes2,3, PhD

1Manchester Academic Health Sciences Centre, Greater Manchester Mental Health Foundation Trust, The University of Manchester, Manchester, United Kingdom

2Institute of Psychiatry, Psychology and Neurosciences, King's College London, London, United Kingdom 3South London and Maudsley NHS Foundation Trust, London, United Kingdom

Corresponding Author:

Shon Lewis, MD

Manchester Academic Health Sciences Centre

Greater Manchester Mental Health Foundation Trust

The University of Manchester

Faculty of Biology, Medicine & Health

Oxford Road

Manchester, M13 9PL

United Kingdom Phone: 44 1613067944 Fax: 44 1613067945

Email: shon.lewis@manchester.ac.uk

Abstract

Background: Improving recovery from acute symptoms and preventing relapse are two significant challenges in severe mental illness. We developed a personalized smartphone-based app to monitor symptoms in real time and validated its acceptance, reliability, and validity.

Objective: To assess (i) acceptability of continuous monitoring to SMI patients and health professionals over 3 months; (ii) impact of active self-monitoring on positive psychotic symptoms assessed at 6 and 12 weeks; and (iii) the feasibility of detecting early warning signs of relapse.

Methods: The active symptom monitoring smartphone app was built into an end-to-end system in two NHS Trusts to enable real-time symptom self-monitoring and detection by the clinical team of early signs of relapse in people with severe mental illness. We conducted an open randomized controlled trial of active symptom monitoring compared to usual management to assess: (i) acceptability and safety of continuous monitoring over 3 months; (ii) impact of active self-monitoring on positive psychotic symptoms assessed at 6 and 12 weeks; (iii) feasibility of detecting early warning signs of relapse communicated to the healthcare staff via an app streaming data to the electronic health record. Eligible participants with a Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) diagnosis of schizophrenia and related disorders, and a history of relapse within the previous two years were enrolled from an early intervention team and a community mental health team.

Results: Of 181 eligible patients, 81 (45%) consented and were randomized to either active symptom monitoring or management as usual. At 12 weeks, 90% (33/36) of those in the active monitoring group continued to use the system and exhibited an adherence rate (defined as responding to >33% of alerts) of 84% (30/36}. Active symptom monitoring was associated with no difference on the empowerment scale in comparison to the usual management group at 12 weeks. The pre-planned intent-to-treat analysis of the primary outcome, a positive score on the Positive and Negative Syndrome Scale (PANSS) scale, showed a significant reduction in the active symptom monitoring group over 12 weeks in the early intervention center. Alerts for personalized early warning signs of relapse were built into the workflows of both NHS Trusts, and 100% of health professional staff used the system in a new digital workflow. Qualitative analyses supported the acceptability of the system to participants and staff.

Conclusions: The active smartphone monitoring system is feasible and was accepted by users in a 3-month study of people with severe mental illness, with surprisingly high levels of adherence. App use was associated with psychotic symptom improvement

https://www.jmir.org/2020/8/e17019

XSL-FO

in recent-onset participants, but not those with longstanding illness, supporting the notion of improved self-management. When built into clinical management workflows to enable personalized alerts of symptom deterioration, the app has demonstrated utility in promoting earlier intervention for relapse.

Trial Registration: ISRCTN Registry ISRCTN88145142; http://www.isrctn.com/ISRCTN88145142

(JMed Internet Res 2020;22(8):e17019) doi: 10.2196/17019

KEYWORDS

digital; smartphone; m-health; psychosis; mental health; self management

Introduction

Severe mental illnesses such as schizophrenia often run a relapsing, lifelong course. Persons with severe mental illness have two primary goals: to improve the speed and quality of their recovery and to prevent future relapse. Following the first episode, 70% will have at least one relapse during the next five years [1]. Despite the rise of community care, 40% of the costs of care for a person with severe mental illness are on unplanned inpatient care for relapse [2]. In standard UK practice, contact with health professionals typically occurs only once every 2-6 weeks, so that early signs of relapse are usually picked up too late to enable prompt intervention. Early warning signs of relapse usually comprise the emergence of a mixture of dysphoric symptoms such as anxious mood, and then attenuated psychotic symptoms, appearing over 1-5 days, with insight usually retained until the day of relapse [3].

We developed a smartphone-based platform in 2010 (ClinTouch) to help persons with severe mental illness to manage their symptoms and prevent relapse. Randomized feasibility trials showed this method of active symptom monitoring to be safe, feasible, and acceptable to people with severe mental illness [4,5]. Users with severe mental illness preferred the smartphone app to an equivalent SMS-based version, which took longer to complete (mean 326 seconds versus 68 for the smartphone app [6]).

Having demonstrated the proof of concept, we integrated the standalone smartphone system (ClinTouch) via an application programming interface (API) into NHS Trust information and communication technology platforms. This integration enabled the streaming of summary information into electronic health records, enabling health professionals to track current symptoms on desktops at the team base and receive personalized alerts when symptoms exceeded a pre-agreed threshold.

This report describes an open randomized controlled trial of smartphone-based active symptom management versus usual care to assess the (i) acceptability and safety of continuous monitoring in persons with severe mental illness and health professionals over 3 months, (ii) impact of active self-monitoring on positive psychotic symptoms assessed at 6 and 12 weeks, and (iii) feasibility of detecting early warning signs of relapse.

Methods

Study Design

The trial of ClinTouch active symptom management versus management as usual was a two-center, open, randomized controlled trial at the NHS Mental Health Trusts in Manchester and South London. Software development, beta testing, and prior cohort and smaller randomized trials had used an experience-driven design process in which service users with severe mental illness were involved in all stages of the design and development of the app, its functionality, and its standard operating procedures. Health professionals were included in design issues where they related to the use of the system within routine practice and in the design of new, digitally-enabled workflows. In preparing for the current trial, 6 focus groups were conducted and audiotaped, including a total of 23 service users, 5 carers, and 30 healthcare staff. Qualitative in-depth interviews were conducted with 19 service users, 6 carers, and 17 staff. A Service User and Carer advisory group met quarterly throughout the project and provided advice on study design, information for participants, and related issues.

The personalized smartphone app triggers the user to rate their symptoms several times a day, and wirelessly uploads these in real time to a secure central server. An audio cue triggered semi-randomly 2-4 times a day reminds the user to complete a set of 12-14 branching items about current symptom severity using a touchscreen slider. A graphical summary of how symptoms fluctuate over time is assembled and displayed on the handset. By conducting face-to-face interview assessments using the gold standard Positive and Negative Syndrome Scale (PANSS) [7] before and after one week of 4 times daily ClinTouch assessment, we confirmed the validity of the self-reported items. Core psychotic symptom and mood items showed moderate to strong (r>0.6) correlations between the in-person and self-report methods [5]. Non-core, behaviorally assessed items such as negative symptoms showed weaker correlations.

The trial was approved by the South Birmingham NHS Research Ethics Committee (14/WM/0045). The trial was registered with the National Institute of Health Research CRN portfolio: 16361, and ISRCTN 88145142. The Medicines and Healthcare Regulatory Agency elected not to designate ClinTouch as a medical device as deployed for the trial.

Participants

One community clinical team from each Trust participated. In England, community mental health teams serve a geographically defined catchment area. All Trusts use electronic patient record systems. Management of individual service users is coordinated by a mandatory care coordinator, usually with a nursing background. Each team has one consultant psychiatrist working with psychologists and other mental health professionals. Teams typically have caseloads of 200-400 people with severe mental

https://www.jmir.org/2020/8/e17019

XSL-FO

illness, with 20-30 care coordinators. In the South London Trust, an Early Intervention for Psychosis (EIP) team was selected and in the Manchester Trust, a Community Mental Health Team (CMHT). NHS EIP teams are configured to manage care for people in the first three years after the first psychotic episode, after which care is transferred to a CMHT. Different types of teams were chosen a priori to investigate the effect of duration of illness on any response to digital treatment. Recruitment took place between February 2014 and May 2015.

Participant inclusion criteria were: (i) operational Diagnostic and Statistical Manual 8th Edition DSM-IV [8] diagnosis of schizophrenia and related disorders; (ii) aged 16-65; (iii) one or more psychotic episodes in the previous 2 years, including the first psychotic episode. Exclusion criteria were: (i) unable to speak English; (ii) unable to give informed consent. Patients who met these criteria were identified separately in the two clinical teams.

Randomization

Participants were allocated by computer using randomized, permuted blocks to one of two groups: active symptom monitoring plus management as usual, or management as usual alone, each for 12 weeks. No stratification was used.

Procedure/Intervention

There were two linked interventions. Active symptom monitoring with feedback to participants was aimed at encouraging self-management of symptoms. Alerts were fed back to the care coordinator when personalized early warning sign thresholds were exceeded, allowing very early intervention.

The ClinTouch active symptom management system was integrated into the electronic health record (EHR) platform of the Manchester NHS Trust via an application programming interface (API). The API instructed the EHR to retrieve data from the ClinTouch dashboard, including a list of participants using the ClinTouch system and any alerts associated with them. The EHR system displayed information relating to the ClinTouch data within the individual patient record. Care coordinators and clinicians were given secure individual logins to the ClinTouch system, enabling them to view the data on a desktop along with graphs of symptom changes over time. The EHR provider for the South London Trust denied API access. To compensate for this lack of access, an automated email was sent to the appropriate care coordinator whenever a patient alert was raised. The email solution was also employed in the Manchester Trust.

Frontline clinical staff (n=42) were trained in the use of ClinTouch. For patients randomized to the experimental group, the relevant care coordinator delivered training in the use of the handset. Either the Android app was installed on the participant’s phone, or a preconfigured Samsung Galaxy smartphone was provided on loan for the duration of the study. The branching items covered positive psychotic symptoms, anxiety, and mood as validated against the PANSS scale in previous studies. Semi-random twice-daily auditory cues from the handset prompted symptom data collection and wireless upload. The system was then used for 12 weeks in the context of the preexisting care plan modified for the ClinTouch algorithms.

Care coordinators then determined the criteria for each participant’s early warning signs, using previous electronic patient records for reference. The early warning sign threshold was set by assigning a score of 0-3 (low through high) to each symptom according to how relevant it is for that participant’s relapse signature based on previous experience. The alert algorithm was constructed so that symptoms scored as 1 collectively comprised 20% of the total early warning sign score, those scoring 2 comprised 30%, and those scoring 3 comprised 50%. An alert was generated if the total score for a single datapoint rose to 40% higher than the baseline defined as the mean score for the first 3 days of recording, or 25% higher across two consecutive data points. Operationally, this was done as part of the standard crisis planning meeting.

Standard operating procedures were established for recording and handling adverse events. Technical measures to ensure data privacy and patient confidentiality followed industry-standard best practices, and all data communications between app and server used encrypted channels. Data were handled per the UK Data Protection Act 1998.

Follow-up and Assessments

Participants were assessed in an in-person interview at baseline, then 6 and 12 weeks after randomization. Research assistants trained to criterion inter-rater reliability undertook participant assessments.

Feasibility and acceptability outcomes in the experimental group were two-fold. The client-centered outcomes included the proportion of eligible clients consenting to a trial of ClinTouch active symptom management. We predicted that 50% would remain in follow-up for 12 weeks. We predicted that 50% of participants would complete >33% of all possible symptom self-ratings over the 12-week trial. The clinical team outcomes included the proportion of all care coordinators accessing patients’ online symptom data. Adverse effects were routinely monitored during the weekly telephone support calls to participants.

Primary efficacy endpoints over 12 weeks included (i) Score on the positive symptom subscale of the PANSS, (ii) user empowerment from interviews, and the Empowerment Rating Scale [9]. Secondary efficacy outcomes were (i) Calgary Depression Scale [10], (ii) Global Assessment of Functioning scale (GAF) [8], and (iii) health-related quality of life, the EuroQol 5D (EQ5D) [11]. These face-to-face interviews were recorded in hard copy versions of the rating scales and the data stored securely in accord with Medical Research Council guidance.

In order to gain an estimate of how frequently clinical staff recorded episodes of possible early warning signs independently of the active symptom monitoring system, transcripts of electronic care records for the 12 weeks of the trial plus a further 4 weeks were anonymized and any reference to randomized treatment redacted. These were then rated independently by two experienced clinicians (SL, RH) for the documented occurrence

of emergent symptoms, which met early warning criteria of documented worsening of psychotic symptoms.

Qualitative interviews were conducted in a subsample of those declining to participate and those allocated to ClinTouch at exit.

Statistical Analysis

The effect of ClinTouch-enhanced monitoring on PANSS Positive Subscale totals at follow-up was examined using analysis of covariance (ANCOVA), including allocation group and site (Manchester or London) as cofactors and baseline scores as a covariate, using Stata 14.1 (College Station). The teams at each site were selected purposely so that differences in response between young, recent onset participants (London) and older, more chronically unwell participants (Manchester) could be examined. Sensitivity analyses examined the effect of demographic variables (covariates were sex, age, level of qualifications, ethnic minority status, living independently, being single, unemployed, in current psychotherapy or abusing alcohol) using backward stepwise elimination of associations of P>.20. A comparison of individual general linear models for the two sites was pre-planned to examine the likely differences. Finally, secondary analyses of other PANSS subtotals and total were conducted in the same way as the primary analysis.

The sample size was calculated based on a 50% reduction in early warning signs in the experimental treatment arm over 12

Lewis et al weeks, from 40% to 20%. Assuming a 10% drop out rate, a sample size of 72 would have 80% power to detect this difference with a one-sided alpha of 0.2, as recommended for a feasibility trial. The analysis was by intent to treat ANCOVA using STATA, with data at baseline, then 6 and 12 weeks.

Results

Recruitment and Feasibility

Of 181 eligible service users approached, 81 (46%) consented to participate and were randomized to either ClinTouch-enhanced management or management as usual (see Tables 1 and 2, Figure 1). There were substantial demographic differences between sites (see Table 2), as intended and expected. The CMHT participants (Manchester) were older and chronically unwell (mean 46 years; median 2.5 hospital admissions, IQR 1 to 4) than the EIT participants (mean 26 years; London: median 1 admission, IQR 0 to 1). Of those 40 who were randomized to the ClinTouch-enhanced management arm, 38 (95%) stayed in the trial for 12 weeks. Of these 38, acceptable adherence as defined by responding to at least 33% of beep alerts (four-item sets per day) was 84%, good adherence (greater than 50% of alerts) was 60%. Healthcare professionals (care coordinators) used ClinTouch-enhanced management in 100% of cases, accessing ClinTouch data an average of 24 times per patient.

Table 1. Demographic data by treatment group.

Descriptor

ClinTouch enhanced monitoring plus standard care

Standard care

Number

40

41

Age (years), mean (range)

33.7 (21-61)

35.3 (20-68)

Sex - female

11

16

Ethnicity - white

20

23

Ethnicity - black/black British

17

15

Ethnicity - other

3

3

Table 2. Demographic data by site.

Descriptor

Community mental health (Manchester)

Early psychosis (London)

Number

37

44

Active symptom monitoring treatment arm

18

22

Age (years), mean (range)

46.1 (21-68)

26.1 (19-36)

Sex - female

14

13

Ethnicity - white

31

12

Ethnicity - black/black British

6

26

Ethnicity - other

0

6


Safety

Adverse effects were routinely monitored during weekly telephone support calls. Of 38 participants who completed 12 weeks of the trial, three (8%) reported significant events: 1 reported increased anxiety prompted by questions; 1 reported increased irritation due to the alert beeps, and 1 had their charger explode. All 3 continued to complete the 12 weeks in the trial.

Clinical efficacy

There were no substantial differences in symptom severity at the point of randomization between those allocated to ClinTouch-enhanced monitoring or standard care (Table 3). On the primary efficacy outcomes, there was no significant difference between groups in PANSS Positive total after 6 or 12 weeks, nor were there significant differences in secondary outcomes (Table 3). Sensitivity analysis showed that including demographic variables made no substantial difference to the allocation group's coefficient or significance.

The planned analyses of each site separately demonstrated different outcomes in the different services. Although there were no significant differences between ClinTouch-enhanced monitoring and control participants in Manchester (apart from a difference in depression scores identifiable at baseline and persisting without significant alteration during the trial; Table 4), findings in London were different (Table 5). There was a significant reduction in positive symptoms after 12 weeks of ClinTouch-enhanced monitoring in the early psychosis subsample (adjusted mean difference -3.04; CI -5.49, -0.59; P=.016. Although there was a significant site-by-group interaction for PANSS total (Supplementary Table 1; P=.003), indicating a significantly lower PANSS total after 12 weeks of ClinTouch-enhanced monitoring in the early psychosis center, this benefit was not in itself significant (adjusted mean difference -5.83; CI -14.14, 2.48; P=.164 2-tailed). There were no other significant site-by-group differences. In addition to the conventional rating scales, the ClinTouch device provided real-time individual active symptom data, which indicated that over 12 weeks, all symptoms except one declined in mean severity. Severity of hallucinations decreased by 29%.

The frequency of early warning signs, as documented in electronic patient records, was 33% in the CEM group and 46% in the control group over 12 weeks, after excluding 8 cases where records were too scant to be rated. The actual performance of this early prototype of the Early Warning Signs algorithm was suboptimal, in terms of the accuracy of ClinTouch alerts versus early warning signs as contemporaneously documented in the electronic patient record. Sensitivity was 75%, specificity 8%, giving a positive predictive value of 29%.

Table 3. Clinical Measures at baseline, 6, and 12 weeks by allocation group.

Scale and visit

CareLoop enhanced monitoring, mean (SD)

Management as

Usual, mean (SD)

Adjusted mean differencea

95% CI

P value

2-tailed

P value

1-tailed

Intercept site*trial arm P

PANSSb Total

Baseline

72.9 (14.8)

76.8 (17.4)

Weeks 6

70.7 (17.0)

73.9 (20.7)

-0.47

-6.47 to 5.53

.874

.44

.46

Week 12

64.5 (15.7)

69.3 (20.7)

-1.93

-7.50 to 3.64

.492

.25

.003

PANSS Positive

Baseline

18.8 (5.4)

18.3 (5.7)

Weeks 6

17.3 (6.2)

17 (6.2)

-0.37

-2.35 to 1.60

.708

.35

.34

Week 12

16 (5.3)

16.7 (6.2)

-1.13

-3.12 to .87

.264

.13

.057

PANSS Negative

Baseline

18.8 (4.3)

18.3 (5.5)

Weeks 6

16.1 (4.4)

18.2 (5.7)

-0.44

-2.18 to 1.30

.616

.31

.75

Week 12

15 (4.4)

17.1 (5.6)

-0.69

-2.51 to 1.15

.462

.23

.53

PANSS General

Baseline

38.2 (8.7)

40 (9.2)

Weeks 6

37.4 (9.5)

38.7 (10.9)

-0.17

-3.70 to 3.37

.994

.47

.64

Week 12

33.5 (8.6)

35.5 (10.7)

-0.79

-3.86 to 2.29

.611

.31

.38

ERSc Total

Baseline

86.3 (7.4)

81.4 (7.8)

Weeks 6

85.4 (7.6)

81.6 (10.3)

0.58

-2.99 to 4.15

.748

.37

.47

Week 12

86.5 (11.9)

83.6 (8.1)

-0.05

-4.35 to 4.25

.983

.49

.32

EQ5Dd Total

Baseline

8.8 (3.1)

9.6 (4.1)

Weeks 6

9.2 (3.4)

8.8 (4.2)

-0.29

-2.43 to 1.85

.286

.14

.29

Week 12

8.0 to 4.1

8.4 (3.8)

0.15

-1.23 to 1.53

.812

.41

.15

CDSe Total

Baseline

5.8 to 4.6

8.1 (5.6)

Weeks 6

6 to 4.5

7.3 (5.2)

0.29

-1.30 to 1.90

.712

.36

.03

Week 12

4.6 to 3.7

6.5 (4.8)

-0.67

-2.24 to 0.90

.4

.20

.83

GAFf

Baseline

49.7 to 14.9

49.3 (11.8)

Weeks 6

49.2 to 14.5

47.7 (16.7)

-0.62

-6.45 to 5.21

.595

.23

.90

Week 12

51.8 to 13.7

52.2 (16.2)

-2.65

-8.38 to 3.07

.850

.43

.30

aFollow-up differences adjusted for baseline scores and the main effect of site. bPANSS: Positive and Negative Syndrome Scale

cERS: Empowerment Rating Scale

dEQ5D: EuroQol-5D

eCDS: Calgary Depression Scale

f

GAF: Global Assessment of Functioning

https://www.jmir.org/2020/8/e17019

XSL-FO

Table 4. Clinical measures at baseline, 6 weeks, and 12 weeks: community team sample.

Scale and visit

CareLoop enhanced monitoring, mean (SD)

Management as usual, mean (SD)

Adjusted mean differencea

95% CI

P value,

2-tailed

PANSSb Total

Baseline

72.73 (11.71)

78.32 (19.02)

Weeks 6

69.47 (17.43)

72.68 (22.53)

1.30

-8.98 to 11.58

.80

Week 12

57.84 (14.23)

68.59 (22.22)

-5.83

-14.14 to 2.48

.16

PANSS Positive

Baseline

19 (4.22)

19.36 (6.12)

Weeks 6

16.42 (5.67)

17.41 (6.65)

-0.95

-3.91 to 2.01

.52

Week 12

14.11 (4.10)

17.18 (6.27)

-3.04

-5.49 to -0.59

.02

PANSS Negative

Baseline

16.09 (4.21)

18.77 (5.52)

Weeks 6

15.42 (4.50)

17.32 (6.12)

0.004

-2.83 to 2.92

.98

Week 12

13.47 (4.58)

16.45 (6.10)

-0.76

-3.36 to 1.85

.56

PANSS General

Baseline

37.64 (7.49)

40.18 (10.13)

Weeks 6

37.63 (10.12)

37.95 (11.46)

1.43

-4.51 to 7.36

.63

Week 12

30.26 (8.55)

34.95 (11.15)

-2.67

-7.46 to 2.13

.27

ERSc Total

Baseline

86.73 (5.59)

83.27 (6.53)

Weeks 6

85.74 (5.69)

83.68 (8.16)

0.13

-4.18 to 4.43

.95

Week 12

86.26 (6.15)

85.91 (8.07)

-1.26

-5.80 to 3.29

.58

EQ5Dd Total

Baseline

8.09 (2.81)

8.64 (3.09)

Weeks 6

6.91 (3.53)

8.23 (3.16)

1.00

-0.78 to 2.79

.26

Week 12

6.45 (3.49)

7.00 (2.31)

0.33

-1.35 to 0.83

.20

CDSe Total

Baseline

5.82 (4.16)

8.5 (5.86)

Weeks 6

6.63 (4.21)

6.73 (4.46)

1.60

-0.486 to 3.67

.13

Week 12

4.21 (3.03)

6.27 (4.38)

-0.85

-2.94 to 1.25

.42

GAFf

Baseline

42.38 (12.04)

38.4 (6.42)

Weeks 6

44.47 (13.29)

41.91 (16.25)

-0.11

-9.40 to 9.17

.98

Week 12

48.47 (11.75)

45.14 (18.93)

-1.09

-10.05 to 7.00

.80

aFollow-up differences adjusted for baseline scores. bPANSS: Positive and Negative Syndrome Scale cERS: Empowerment Rating Scale dEQ5D: EuroQol-5D

eCDS: Calgary Depression Scale

f

GAF: Global Assessment of Functioning

https://www.jmir.org/2020/8/e17019

XSL-FO

Table 5. Clinical measures at baseline, 6, and 12 weeks: First episode psychosis sample.

Scale and visit

CareLoop enhanced monitoring, mean (SD)

Management as usual, mean (SD)

Adjusted mean differencea

95% CI

P value, 2 tailed

PANSSb Total

Baseline

73.11 (18.16)

75.00 (15.72)

Weeks 6

72.06 (16.86)

75.44 (18.77)

-2.84

-8.81 to 3.12

.34

Week 12

71.50 (14.31)

70.11 (19.24)

2.69

-4.81 to 10.19

.47

PANSS Positive

Baseline

18.44 (6.66)

17.16 (5.1)

Weeks 6

18.24 (6.82)

16.56 (5.79)

2.67

-2.48 to 3.01

.84

Week 12

18.06 (5.86)

16.06 (6.35)

1.10

-2.16 to 4.35

.498

PANSS Negative

Baseline

15.83 (4.48)

17.95 (5.69)

Weeks 6

16.76 (4.28)

19.28 (5.04)

-1.03

-3.01 to .94

.30

Week 12

16.5 (3.73)

18.06 (4.83)

-0.50

-3.09 to 2.10

.70

PANSS General

Baseline

38.83 (10.17)

39.84 (8.34)

Weeks 6

37.06 (9.13)

39.61 (10.32)

-2.32

-5.85 to 1.23

.19

Week 12

36.94 (8.91)

36.11 (10.44)

1.33

-2.60 to 5.26

.495

ERSc Total

Baseline

85.83 (9.32)

79.26 (8.84)

Weeks 6

85 (9.52)

78.94 (12.25)

1.40

-4.73 to 7.53

.65

Week 12

86.72 (16.05)

80.72 (7.43)

1.63

-6.25 to 9.50

.68

EQ5Dd Total

Baseline

9.5 (3.59)

11.21 (4.60)

Weeks 6

9.22 (4.25)

10.42 (4.86)

0.10

-2.48 to 2.69

.94

Week 12

8.06 (7.38)

6.116 (5.17)

-1.40

-5.72 to 2.02

.51

CDSe Total

Baseline

5.72 (5.21)

7.68 (5.23)

Weeks 6

5.29 (4.81)

8.06 (6.00)

-1.36

-0.98 to 3.69

.25

Week 12

5.06 (4.41)

6.83 (5.48)

-0.63

-1.81 to 3.07

.61

GAFf

Baseline

56.94 (13.12)

53.84 (12.30)

Weeks 6

52.64 (14.52)

51.56 (16.79)

-1.19

-8.51 to 6.15

.74

Week 12

54.11 (14.65)

57.17 (10.78)

-5.09

-12.38 to 2.20

.17

aFollow-up differences adjusted for baseline scores. bPANSS: Positive and Negative Syndrome Scale cERS: Empowerment Rating Scale dEQ5D: EuroQol-5D

eCDS: Calgary Depression Scale

f

GAF: Global Assessment of Functioning

https://www.jmir.org/2020/8/e17019

XSL-FO

Discussion

We conducted an open randomized controlled trial of active symptom monitoring compared to usual management in people with serious mental illness to assess over 12 weeks the (i) acceptability and safety of continuous monitoring, (ii) impact of active self-monitoring on positive psychotic symptoms, and (iii) feasibility of detecting early warning signs of relapse communicated to the healthcare staff via an API allowing data to be streamed into the EHR.

A systematic review has suggested that smartphone apps may be helpful in the management of mental health disorders such as depression [12]. Nonetheless, almost none of the publicly available mental health apps have good quality data concerning safety and efficacy [13]. The real-time digital approach used in this study holds several advantages over routine clinical assessment. It reduces the confounding effects of retrospective recall bias, forgetting, and averaging in symptom appraisal. It allows the context of symptom changes to be assessed and increases patient involvement in continuing care through participation in symptom and progress monitoring. It may also enable a degree of symptom self-management via a trusted and ubiquitous, ever-present personal device.

The trial demonstrated several things. The active symptom monitoring intervention was safe and acceptable: 45% of the eligible sample agreed to enter the trial. Furthermore, and importantly, of those using the ClinTouch-enhanced monitoring system, 90% continued to use it regularly at 3 months. In these patients, adequate adherence was 84%, defined as responding to >33% of item prompts. On pre-planned intent-to-treat analysis, the primary outcome of positive symptom score on the PANSS scale showed a significant reduction in the ClinTouch group over 12 weeks only in the early intervention center. The larger therapeutic effect in the early psychosis participants was not due to the severity or adherence differences between the two subsamples. It may be that, as has been shown with pharmacological and psychological treatments for psychosis, the therapeutic effect is larger earlier in the course of the disorder.

We have demonstrated from a software perspective that we can build an algorithm into the ClinTouch app to provide an alert


Lewis et al when symptoms start to worsen. An API allowed this to be built into the electronic patient record system in one Trust. With symptom data streamed into the EHR system, health professionals could view it on a secure desktop at the team base. Alerts for early warning signs were built into the workflows of the two NHS Trusts, and 100% of health professional staff used the system to access symptom data and alerts in a new digital workflow. Qualitative analyses supported the acceptability of the system to participants and staff.

There were limitations to the trial. In the second Trust, the commercial provider of the EHR did not comply with the study, indicating a potential barrier to full scale roll out in the NHS where Trusts have a range of different commercially provided EHR platforms. Another limitation was that, at the time of the trial (2014-2016), the ClinTouch app was only available for the Android operating system. In addition, the accuracy of the early prototype in detecting EWS was limited by our focus being mainly on operability. Case record documentation of EWS was often scanty, proving to be an inadequate gold standard. Artifacts in functionality were identified for improvement, such as alerts being mistimed if the user was temporarily in an area without a wireless network. Subsequent versions are proving more refined. Further work is now taking place to refine the alert algorithm through robust risk prediction modeling in order to increase its sensitivity and specificity and improve the effectiveness of promoting early intervention by clinical teams to improve patient outcomes.

In conclusion, the active smartphone monitoring system is feasible and acceptable over three months to users with severe mental illness, with surprisingly high levels of adherence both from users and health professionals. It was associated with psychotic symptom improvement in patients with recent-onset psychosis, and supports the notion of improved self-management in those with first episode psychosis. In terms of implications for clinical practice, digital health interventions appear to hold considerable promise in the management of people with psychosis. Smartphone-based active symptom monitoring can be built into EHR systems and regular clinical workflows and allow preventive, personalized care, especially if combined in with added digital functionality such as medication management and physical health monitoring.


Acknowledgments

This study was funded by Medical Research Council Developmental Clinical Studies grant MR/K015516. The Medical Research Council had no role in the study design, data collection, analysis, interpretation, or writing of the report. The corresponding author had full access to all data and had final responsibility for the decision to submit for publication.

Conflicts of Interest

SL is the Director of Affigo, a not for profit social enterprise digital company and the Medical Director of Xenzone (remunerated), a digital counselling company. JA, CS-P, and PW are Directors of Affigo CIC, which is a community interest company set-up in 2015 to make ClinTouch more widely available to mental health Trusts. The IP for ClinTouch has been assigned to Affigo. Our company registration number is 09928775.

Multimedia Appendix 1

CONSORT eHEALTH checklist (V 1.6.1).

https://www.jmir.org/2020/8/e17019

XSL-FO

[PDF File (Adobe PDF File), 957 KB-Multimedia Appendix 1]

References

Abbreviations

ANCOVA: analysis of covariance

API: application programming interface

CDS: Calgary Depression Scale

CMHT: Community Mental Health Team

DSM-IV: Diagnostic and Statistical Manual of Mental Disorders, 4th Edition

EHR: electronic health record

EIP: Early Intervention for Psychosis

EQ5D: EuroQol 5D

GAF: Global Assessment of Functioning

PANSS: positive and negative syndrome scale

https://www.jmir.org/2020/8/e17019

XSL-FO

Edited by G Eysenbach; submitted 13.11.19; peer-reviewed by H Mehdizadeh, T Timmers; comments to author 27.12.19; revised version received04.02.20; accepted 16.02.20; published 13.08.20

Please cite as:

Lewis S, Ainsworth J, Sanders C, Stockton-Powdrell C, Machin M, Whelan P, Hopkins R, He Z, Applegate E, Drake R, Bamford C, Roberts C, Wykes T

Smartphone-Enhanced Symptom Management In Psychosis: Open, Randomized Controlled Trial

J Med Internet Res 2020;22(8):e17019

URL: https://www.jmir.org/2020/8/e17019

doi: 10.2196/17019

PMID: 32788150

©Shon Lewis, John Ainsworth, Caroline Sanders, Charlotte Stockton-Powdrell, Matthew Machin, Pauline Whelan, Richard Hopkins, Zhimin He, Eve Applegate, Richard Drake, Charlie Bamford, Chris Roberts, Til Wykes. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.08.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

Social Psychiatry and Psychiatric Epidemiology https://doi.org/1Q.1QQ7/sQG127-G19-G1681-2

ORIGINAL PAPER


A pilot digital intervention targeting loneliness in young people with psychosis

Michelle H. Lim1,2 • John F. M. Gleeson3 • Thomas L. Rodebaugh4 • Robert Eres1,2 • Katrina M. Long1 • Kit Casey2 • Jo-Anne M. Abbott5 • Neil Thomas2 • David L. Penn3,6

Received: 22 November 2Q18 / Accepted: 25 February 2Q19

© Springer-Verlag GmbH Germany, part of Springer Nature 2Q19

Abstract

Purpose Loneliness has been identified as a significant challenge for people with psychosis. Interventions targeting loneliness are lacking but adopting a positive psychology approach may reduce loneliness, promote well-being, and support meaningful social interactions. Together with youth mental health consumers, we developed a digital smartphone application (app) called +Connect, which delivers positive psychology content daily for 6 weeks.

Materials and methods Twelve participants diagnosed with a psychotic disorder were recruited from early psychosis services. Loneliness was assessed pre-intervention, post-intervention, and 3-month post-intervention. Acceptability, feasibility, and usability were measured post-intervention, including a semi-structured interview on the user’s experience of +Connect. Results We found evidence for the feasibility of +Connect. All but two participants completed the +Connect program, completing 95% (40.10 out of 42 days) of the program. Furthermore, 66.67% (8 out of the 12 participants) remained engaged with the program 3-months post-intervention. Our data indicates preliminary evidence that +Connect may reduce loneliness, with scores from pre-intervention (M = 50.00, SD = 8.47) to post-intervention (M=48.10, SD = 10.38) and 3-months post-intervention (M=42.89, SD = 7.04). We found that positive reinforcement of in-game rewards and evidence of positive mood changes added to the feasibility of the app. Regarding acceptability, while 10% (1/10 participants) reported not finding +Connect useful or enjoyable, 90% of participants agreed that +Connect helped them to increase their social confidence, enjoy life, look forward to being with other people, and feel more connected with others. Participant interviews supported these results, with participants highlighting the app’s strengths in providing useful information, stimulating self-reflection, fostering positive affect, and encouraging transfer of skills into their social interactions.

Discussion While preliminary findings indicated that +Connect yielded high levels of acceptability and feasibility, it is important to consider that we recruited a small and selected sample of lonely young people. Further iterations of this proof of concept app, which can incorporate participant feedback such preferences for increased personalisation, in-app feedback, and gamification, may allow an opportunity to test an improved version in the future.

Keywords Loneliness • Psychosis • Positive psychology intervention • Digital intervention

Introduction

Jo-Anne M. Abbott was at the National eTherapy Centre, Swinburne University of Technology, Melbourne, Australia at the time of this research.

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s00127-019-01681-2) contains supplementary material, which is available to authorized users.

* Michelle H. Lim

mlim@ swin.edu.au

Extended author information available on the last page of the article Addressing loneliness has been identified as a high priority in helping people who have psychosis [1, 2] but we currently lack evidence-based solutions for this subjective sense of social isolation in this population [3]. Reducing loneliness in young people with psychosis is particularly important because there is increased stigma [4], societal discrimination [5], and a change in social roles (e.g., dropping out of education or work [6]) associated with the onset of psychosis. Most interventions have included enhancing social support and opportunities, improving social skills, or addressing maladaptive social cognition [7] but thus far, positive psychology interventions (PPIs) have been overlooked as an alternative approach in combatting loneliness.

PPIs focus on enhancing the person’s functioning [8] and can be used as complementary approaches within mental health interventions [9]. Rather than aiming to correct deficits, PPIs take a strength-based approach to psychotherapy, which may be more engaging and less pathologizing [1011]. PPIs have been shown to effectively enhance recovery by encouraging an individual to identify positive emotions [12, 13], express gratitude [14], practice kindness [15], set goals [16], and identify and use strengths [14].

PPIs can also be applied to improving personal and social relationships [17]. Several PPIs include exercises such as active-constructive responding; which aim to enhance the individual’s relationship well-being in the context of responding to hearing positive news from others [18-20]. A PPI that promotes positive affect and facilitates the use of positive interpersonal skills could plausibly improve relationship quality [21]. Intervention studies using positive psychology approach have yet to specifically target loneliness [12, 13].

Digital tools are increasingly utilized within mental health treatment but it is important to consider that its success and integration to clinical services may relate to a multitude of factors from the user’s psychiatric symptoms severity to lack of resources (e.g., finances and staff support) [22]. Digital tools have nonetheless been developed and tested for people with psychosis [23-25], either via web-based platforms [26], virtual reality/avatars [27, 28], and smartphone applications, or ‘apps’ [29]. These tools are used for a variety of purposes from monitoring symptoms [30], medication adherence [31], promotion of self-management strategies (e.g., improving coping), provision of psychoeducation [32], and social recovery, such as enhancing access to peers [33].

Pilot studies in smartphone app interventions so far have been found to be acceptable and feasible in people with psychosis, but the acceptability and feasibility criteria set prior to each trial may vary. For example, one study measured acceptability via satisfaction ratings on app features or retention rates, and feasibility, via login frequency, challenge completions, number of social interactions within the app [34]. Another defined acceptability as collecting participant feedback and feasibility was assessed as the program uptake, completion, and attrition. Smartphone technology is being used for a variety of ways in those with psychosis; from assessment and monitoring [35, 36], improving motivation and enhancing support (e.g., PRIME [34, 37]), addressing clinical symptoms and preventing relapse (e.g., Actissist [38]), delivering case management cognitive-behavioural therapy (e.g., Heal Your Mind [39]), and facilitating medication adherence [40]. We extended upon this growing list by developing a smartphone app that can deliver PPI content to young people with psychosis to specifically target loneliness. Targeting loneliness rather than other broader social problems is currently lacking in the literature. This is because researchers have traditionally favoured measuring objective indicators of social connection such as increased social ties [3] over subjective indicators such as loneliness within psychosocial interventions. But more recent evidence has suggested that those with psychosis not only report loneliness as prevalent but also a top challenge for them to overcome [1, 2].

Study aims and hypotheses

The first aim was to develop a pilot digital smartphone application to target loneliness that is feasible and acceptable to young people with early psychosis. We hypothesized that participants would complete at least 70% of the program (equivalent to 30 of the 42 days of content).

The second aim was to develop an app that is usable to young people as one necessary step to reducing the likelihood of poor engagement, a common problem with mental health smartphone apps [41]. We hypothesized that more than 70% of participants would report higher than somewhat in their satisfactory ratings across a series of criteria, including ease of understanding, enjoyment in life, and content helpfulness.

Third, we explored the usability of the app (e.g., functionality, navigation). Next, within an exploratory analysis, we also estimated the plausible effect size of +Connect on loneliness severity using a latent trajectory model. Last, we used a mixed methods approach using quantitative and qualitative data to deepen our understanding of how young people experience +Connect.

Methods

Participants

Twelve individuals aged 17 to 25 years (M = 20.50, SD = 2.65), were recruited from early psychosis services in Melbourne, Australia. Table 1 outlines the demographic information of the participants including clinical

Table 1 Demographics of participants

Demographic variable

M (SD) or %

Gender

25% female

Age

20.50 (1.33)

Ethnicity

Asian Australian or Asian

25%

White (including Caucasian, European, Australian)

66.7%

African Australian

8.3%

Relationship status (% Single)

91.7%

Living status

Residing with housemates

16.7%

Residing at home with immediate family

75%

Residing with relatives

8.3%

Residing with

One other person

25%

Two other people

25%

Three other people

16.7%

Four other people

25%

Five other people

8.3%

Completed education (in years)

DSM V diagnosis'1

12.25 (1.72)

Schizophrenia

50%

Schizoaffective

16.7%

Schizophreniform

16.7%

Psychotic disorder NOS

16.7%

Secondary diagnosis

DSM V social anxiety disorder

25%

DSM V mood episode

16.7%

Both social anxiety and mood episode

25%

Neither social anxiety nor mood episode

33.3%

NART FS IQb

109.67 (7.42)

aDSM V refers to the diagnostic and statistical manual of mental disorders 5

bNART FS IQ refers to the National Adult Reading Test full scale intelligence quotient

diagnosis. The study inclusion criteria were: (1) aged 16-25; (2) current DSM V diagnosis of psychotic disorder (i.e., Schizophrenia, Schizophreniform, Schizoaffective Disorder, Delusional Disorder or Psychotic Disorder Not Otherwise Specified) as assessed by the SCID-5; (3) UCLA Loneliness Scale score > 38;1 (4) identified a desire to connect with others; (5) currently engaged with a mental health service, general practitioner or organization with consent for researchers to contact service in case of risk; (6) owned a compatible smartphone. The study exclusion criteria were presence of one of the following in the past month: (1) acute psychotic symptoms;14 15 (2) moderate or severe risk issues, i.e., deliberate self-harm and suici-dality;16 (3) psychiatric hospitalisation; (4) substance abuse or dependence; (5) known Axis II personality disorder; (6) inability to read or comprehend English (NART score < 70 or failure on reading comprehension test).

Development of +Connect digital smartphone application intervention

We first translated positive psychology concepts traditionally delivered via face-to-face group program17 into easy to understand, youth-friendly digital materials to assess the acceptability and feasibility of the PPI content. The aim of the content was to assist individuals to identify and harness their personal strengths, and to learn and practice positive interpersonal skills that could strengthen their current relationships. Themes included eliciting positive emotions, as well as showing kindness and reciprocity within relationships (see Online Resource 2 for more details). We added one additional theme, Social Fears, which addresses social anxiety. This was added to the program because of previous research which indicated a reciprocal relationship between loneliness and social anxiety over time [42].

In 2015-2017, we conducted a series of focus groups with young people aged 18-25. These groups comprised of young people with no mental ill health, young people with high prevalence disorders as well as those with serious mental illnesses. We opted to develop a smartphone app over other digital platforms because of its mobility and accessibility [43-45] and preferences obtained from focus groups. The smartphone app format meant that information was delivered more frequently but in a concise format as opposed to other conventional modes of psychotherapy (e.g., face-to-face) which may require more effort [34]. Before the development of the smartphone app, young people were invited to comment on: design (i.e., fonts, colours, layout), functionality (e.g., task completion and gamification), and language (e.g., written task and video content).

To accurately relay socially oriented information and increase engagement in the app, we also opted to deliver content via video material where possible. We developed three types of videos: (1) shared experience videos (SEVs) using young people with lived experiences [46]; (2) expert videos (EVs) featuring therapists [47]; or (3) actor videos (AVs) demonstrating how to elicit positive affect and initiate or maintain positive social interactions [48].

While mobile and accessible, core concepts had to be concise, and content was delivered over 42 days (6 weeks). When the application is opened, participants see a home screen, and are asked to log their mood using a mood evaluation tracker. They then proceed to the tasks which were delivered in one of four ways: (1) via text and images (e.g., an Instagram format); (2) SEVs featuring young people with lived experiences; (3) EVs featuring academics introducing core concepts; or (4) AVs featuring semi-professional actors modelling a range of social behaviours.

All videos were designed to be brief (i.e., 1.21-4.38 min). AV scripts were written by a scriptwriter and reviewed in a series of focus groups with young people with psychosis. This process enabled us to refine the language used and to ensure that the material was youth-friendly and relatable. The interview schedule for SEVs were developed by MHL, JA, and NT. Two independent coders rated the content of each SEV on whether it achieved the aims of the modules (e.g., Gratitude video: to relay that expressing gratitude can feel awkward at first and it is more than saying thank you).

After accessing daily videos, participants were given a task to answer questions (either multiple choice or True/ False format) in relation to the material, taking under a minute to complete. +Connect is gamified18 (e.g., points, challenges, badges) to encourage participant engagement [4149, 50]. Online Resource 1 outlines the content of the +Con-nect app, developed by MHL, JFMG, TLR, and DLP. This table shows when the different modules and tasks were delivered via levels and days. It shows the aim of each module, for example, Level 5, Day 10-12, within Gratitude module, the aim was to introduce an interpersonal focused gratitude exercise, where the content should include the importance of going beyond to say thank you. Specifically, relaying that gratitude is difficult to do and can be confronting, however, doing the exercise can bring people closer. These concepts were delivered by three days of content titled, Gratitude (written content post), The Gratitude Exercise (an actor video) and Showing Gratitude (a shared experienced video). The table also indicates whether a challenge was introduced, and in all cases, challenges were only unlocked after the entire module was delivered (e.g. Day 12 of the Gratitude module).

Materials

Participants attended three research assessments: Time 1 (T1), baseline; Time 2 (T2), post-treatment (after completing at least 33 days of +Connect); and Time 3 (T3), 3-month follow-up (conducted 3 months after the T2 assessment). Each assessment involved clinician-administered measures and self-report questionnaires.19 Participants also completed an interview at T2 in regards to their experience of using the app.

Acceptability, feasibility, and usability A series of questionnaires were created to assess acceptability, feasibility, and usability of +Connect. Participants were asked to rate their level of agreement with a series of statements regarding their experience of using the app, and how they felt after using the +Connect app. Scores were rated on a 5-point Likert-type scale ranging from 1 (Extremely Disagree) to 5 (Extremely Agree). A 20-item questionnaire designed for the study was used to assess how helpful each module was for the participants. Responses were made on a 4-point Likert-type scale ranging from 1 (Not Helpful) to 4 (Very Helpful).

Clinician-administered measures

The Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders (SCID-5-RV; [51]). The SCID-5-RV was administered at only baseline to determine study eligibility and establish clinical diagnoses. Thirty percent of the assessments were randomly selected and independently rated by another coder (KC), with 100% consensus on diagnostic reliability.

The Positive and Negative Syndrome Scale (PANSS; [52]). The PANSS was used to assesses symptoms of schizophrenia. The PANSS is a 30-item clinician-rated symptom severity measure. The items are rated on a 7-point Likert scale ranging from 1 (Absent) to 7 (Extreme severity). The PANSS has demonstrated good internal inconsistency and reliability [53]. Internal consistencies ranged from as = 0.82 to 0.93.

The Calgary Depression Scale for Schizophrenia (CDSS; [54]). The CDSS, a 9-item clinician-rated measure was used to measure depressive symptoms. It was specifically designed for the assessment of depressive symptoms in those with schizophrenia. It has been shown to have adequate reliability and validity [55]. Internal consistencies ranged from as = 0.77 to 0.81.

Social Skills Performance Assessment (SSPA; [56]). We measured social skills as a potential covariate and to ensure that the intervention had no effect on social skills. The SSPA involves two 3-min role-plays with the research assistant on pre-determined social situations (e.g., participant plays the role of a tenant meeting a new neighbor). Role-plays were audio-recorded and two independent trained coders rated the social interactions on nine separate parameters.20 Intraclass correlation coefficients (ICCs) for T1-T3 ranged from acceptable to excellent (Scenario 1; ICCs = 0.85-0.90; Scenario 2: ICCs = 0.82-0.94). Internal consistencies ranged from as = 0.90-0.91 for scenario 1, and 0.94-0.96 for scenario 2.

National Adult Reading Test (NART; [57]). We used the NART at baseline to ensure participants could comprehend the content of +Connect and were also able to obtain a pre-morbid IQ. The task involves reading a list of 50 words presented in increasing difficulty. Each word is irregular with regards to common pronunciation rules, ensuring participants have familiarity with the word rather than solely relying on phonemic decoding.

Self-report measures

The Revised UCLA Loneliness Scale (Revised UCLA-LS; [58]). The UCLA-LS, a 20-item self-report scale, was used as a measure of loneliness severity, employing a 1 (Never) to 4 (Always) Likert type scale. The measure consists of both positively and negatively worded items that assess loneliness (e.g., How often do you feel that you are no longer close to anyone?). The UCLA-LS has been shown to correlate negatively with life satisfaction and perceived social support, thus supporting its convergent validity with related constructs [58]. Internal consistencies ranged from as = 0.91 to 0.94 across time points.

The Social Interaction Anxiety Scale (SIAS)—straightforward items (S-SIAS [59]). The original SIAS is a 20-item self-report questionnaire that measures anxiety-related reactions to different social interactions (e.g., I get nervous if I have to speak with someone in authority). The 17 Straightforwardly-worded items (S-SIAS) were found to be more valid indicators of social interaction anxiety than the reverse-scored items across different samples [60]. For this reason, we used the straightforward items and internal consistencies ranged between as = 0.87 and 0.94.

The Scales of Psychological Well-Being (SPWB; [61]). The SPWB is a 54-item questionnaire that measures eudai-monic well-being across six dimensions: autonomy, positive relations with others, environmental mastery, personal growth, purpose in life and self-acceptance. Items are scored on a 6-point Likert Scale from 1 (Strongly Disagree) to 6 (Strongly Agree). This widely used scale has demonstrated good internal consistency and construct validity [61]. We used an overall SPWB score. Internal consistency for the SPWB ranged from as = 0.81 to 0.87.

Semi-structured qualitative interview Participants were invited to complete a semi-structured interview regarding their experiences using +Connect at T2.21 The interview was transcribed verbatim prior to analysis.

Design and procedures

We employed an uncontrolled single-group design with three time points to assess the feasibility, acceptability, and usability of a digital intervention targeting loneliness. Participants were recruited via case managers working at local early psychosis teams. Potential participants were initially screened via telephone to assess study eligibility. Participants were screened to ensure that they were not acutely unwell, engaging in problematic substance use, or at moderate to high risk. Participants also completed the UCLA Loneliness 3-item Scale (see [62] for more details); those who scored 5 or more and did not meet exclusion criterion were invited to a baseline assessment, during which they completed the remaining measures.

At the baseline assessment, participants provided written informed consent and proceeded to complete the UCLA-LS, NART, SCID-5, and PANSS. Participants were excluded at this point if they met any of the exclusion criteria. Participants who were deemed eligible proceeded to complete the remaining baseline assessments including the CDSS, SSPA, and self-report questionnaires. Participants’ responses were audio-recorded for quality assurance purposes for the SCID-5, PANSS, SSPA, NART, and the semi-structured interview. Participants were not identified by name during these recordings to ensure the data was de-identified.

Once participants had been accepted into the program, research assistants provided an introduction and orientation to the app, specifically, helping them fill in basic demographic questions within the app. Research assistants explained the purpose, design, and functionality of the app, and provided a demonstration of navigating through the different components. As participants progressed through the app, research assistants monitored their progress via a web portal. Participants were contacted once a week as a brief check in either via text message or a phone call. This was to ensure that they could address any technical issues they

Table 2 Loneliness, mental health symptoms, social skills, and wellbeing across time

M (SD)

T1

T2

T3

UCLA-LS

52.58 (9.94)

48.10 (10.37)

42.89 (7.04)

S-SIAS

39.50 (12.46)

29.60 (8.91)

30.75 (4.86)

CDSS

6.00 (4.71)

4.40 (4.22)

2.78 (2.86)

PANSS

40.00 (8.66)

34.60 (5.42)

35.11 (8.04)

SSPA

62.08 (14.99)

67.55 (13.73)

71.56 (12.27)

SPWB

183.82 (33.73)

189.44 (28.62)

212.63 (21.61)

Note: Total N =12, except for T2 where n =10 and T3 where n = 8 due to participants lost to follow-up. T1 refers to baseline, T2 refers to post-treatment, T3 refers to 3 month follow-up. Values are based on observed values

UCLA-LS University of California Loneliness Scale, S-SIAS straightforward items from Social Anxiety Interaction Scale, CDSS Calgary Depression Scale for Schizophrenia, PANSS positive and negative symptom syndrome, SSPA Social Skills Performance Assessment Scale, SPWB Scales of Psychological Well-Being Scale

may have experienced, and to ensure that potential emerging risk issues were identified and addressed. Participants were reimbursed for the completion of each assessment ($15 per hour) and for mobile data based on the number of days they had logged in ($3.33 per day).

Data analytic procedure

The means and standard deviations were calculated for each measure (see Table 2). The feasibility of the intervention was assessed by uptake, attrition, app completion, and retention at T2. Uptake was defined as the number of potentially eligible individuals who attended the T1 assessment. Attrition was defined as non-use of the app for > 3 consecutive days and inability of researchers to contact the participant. App completion was defined as accessing and completing content at least 70% of the content (30 out of the 42 days). Consistent with most pilot interventions and recommendations [63], we also considered Safety, which was operationalized as the incidence of serious adverse events (such as inpatient psychiatric admissions) during the course of the study [64]. Intervention acceptability and app usability were assessed using satisfaction rating questionnaire and participant interviews at T2. Participant interviews were analysed using content and thematic analysis.

To estimate the overall effect of treatment with the greatest precision given the small sample size, we used a latent trajectory model in Mplus [65]. This technique had the advantages of (a) assessing the effect of treatment across the entire study (e.g., not just pre to post) and (b) allowing a summary across variables (i.e., instead of a series of underpowered t-tests or effect size estimates for each variable).

We first present results using the UCLA-LS, which is considered the gold-standard self-report measure for loneliness. To increase precision of the estimate, we examined which other variables were most associated with loneliness at pre-intervention as measured by the UCLA-LS; these were the SPWB and CDSS, which correlated so highly with the UCLA-LS (rs > 0.79) that examining the measures individually could be misleading (i.e., because they appeared to primarily reflect overlapping variance). These measures were put on the same metric (that of standard deviations) and treated as being influenced by a single intercept and a single linear slope. These analyses are, of course, underpowered; nevertheless, if they were to indicate that the intervention produced a harmful effect or close to no effect, such a result would weigh against the acceptability of the intervention.

Results

Feasibility ratings

Twenty-five potential participants identified by their case managers were telephone screened over 18 months of recruitment across three sites. Of this, seven were found to be ineligible for the study; three were uncontactable, two were assessed to be experiencing acute psychotic symptoms, and the last two individuals reported they were no longer interested in participating. Therefore, 18 young people met the initial telephone screening criteria and attended baseline assessments, of whom 12 met the study eligibility. Four out of 18 of participants were excluded due to a having a diagnosis of substance-induced psychosis and one participant was uncooperative and we were unable to make a confirmatory diagnosis. The final participant did not meet criteria for a psychotic disorder, rather their primary diagnosis was a mood disorder with psychotic features.

Hence, only 12 participants were accepted into the study based on the study’s inclusion and exclusion criteria. Two out of 12 participants (16.67%) dropped out during the intervention, one moved away and one did not cite a reason. Participants (i.e., n = 10)9 on average completed 95.47% of the +Connect (M=40.10 days, SD = 3.04), exceeding the a priori criteria of app completion (33 out of 42 days). Eight out of the ten participants also remained engaged in the program 3 months following the end of treatment assessment.

Feasibility qualitative engagement data Of the ten participants, two reported early difficulties integrating the use of the app into their routine, with one describing the app as “sort of a chore” as they “felt like [they weren’t] really working towards anything. ” Nevertheless, these two

Table 3 Participant interview feedback

Theme

Representative quote

Feasibility—app engagement

Relatability of content

“I like the shared experience videos to see that .. normal people like I .. they applied it and got something out of it. So like I can too. . And, and the videos seemed like really realistic.. .and honest”

“I felt all of the sections were relevant. Because I felt like the people in the videos felt at some point”

Gamification

“I really like the badges idea. Because. that really helped me to keep going with the thing instead of seeing like, ‘Oh, I don’t know like 30 videos!’ because then it doesn’t seem that appealing”

Evidence of positive changes

“At the beginning I kept forgetting because it wasn’t part of my routine. but when I did do it, I started seeing kind of like results”

Notifications

“It’d be a couple of days where I’d be like ‘oh **** I haven’t done it yet’. And because the reminder was up on my phone I was like “oh yeah, do that now”.”

Acceptability—app outcomes

Increased positive affect

Improved social interactions

Increased social confidence

“The best thing about the app. it just makes you feel more positive”

“I reckon I’ve become better at talking to people I don’t actually know”

“I think I feel a lot more confident in myself. I think prior to it I was a bit, not shy, but a bit hesitant in social situations”

Intent to apply in future

“I find that there are some things I could apply to real life. And this is what I am planning, or would like to do later on when I have a chance to meet with other patients, or I could use this app to help me to build those new relationships.”

Encouraged learning

“I felt like it was quite an educational app that teaches you many things”

“I like how at the end of the videos there is always a summary that talks about what was discussed in the video and puts it into text, makes it easier to remember the key points”

Acceptability—app outcomes continued

Encouraged self-reflection

“I think it was good that it logged different moods... just to know how I was feeling . different times of the week. And yeah, so I could look back and see what I did and see how I felt in different scenarios”

“It’s good to actually just ruminate on what is your strengths, like what you’re actually good at. I found that writing down some of your strengths and stuff each day before you go to bed was good”

Usability

App design and navigation

“I like the simple set up and layout of the app”

“The layout and that make it really easy for anyone to use even if you’re not a tech-head”

Video quality

“I found the videos very good. I thought they were well made.I thought the acting was actually pretty good”

Task and challenge difficulty level

“It wasn’t complicated at all and you can apply it pretty much on the day, like the skills that you learn and stuff”

“I found the [question] difficulty not too hard, but it’s not too easy at the same time, like you’d have to have watched the video to get a better gauge of the answers.”

“Some [challenges] were hard. Yeah, some were tricky. I’d be like ‘I’m not doing that’....They were just ones. way out of my comfort zone”


participants, as well as five further participants reported that the app eventually became “just another item to do on [their] schedule”. Eight out of 10 participants reported that they were interested in logging into +Connect daily (i.e., agreeing or extremely agreeing). Six out of 10 participants found the app did not create a significant time burden and reported a duration of 1 and 3 min per day in the program, while the remaining 4 out of 10 reported using the app at least 5 or more minutes per day. See Table 3 for more details regarding engagement.

Acceptability quantitative outcome satisfaction and agreement ratings

Moreover, 80% of participants (8 out of 10) agreed that the +Connect app was useful (see Table 4). Ninety percent of participants (9 out of 10) agreed (or strongly agreed) that they enjoyed using the app, that they gained a lot from using +Connect, and that they found the content understandable and relatable. The ratings of the enjoyment of challenges, however, appeared to be split, where five out of 10 participants endorsed a neutral rating and the other five out

Table 4 Post-intervention feasibility, acceptability, and usability ratings of +Connect

Question

Extremely

disagree

Disagree

Neutral

Agree     Extremely

agree

n    %

n%

n%

n % n      %


Feasibility

Interested in signing in             -

Acceptability

-

1

10

1

10

6

60

2

20

Enjoyed using +Connect          -

-

1

10

-

-

7

70

2

20

+Connect was useful             -

-

1

10

1

10

5

50

3

30

Enjoyed content                 -

-

-

-

3

30

6

60

1

10

Understand the ideas              -

-

-

-

1

10

5

50

4

40

Gained a lot                      -

-

1

10

-

-

7

70

2

20

Could relate to content            -

-

-

-

1

10

7

70

2

20

Relate to characters                -

-

-

-

3

30

5

50

2

20

Videos helped with content        -

-

-

-

3

30

2

20

5

50

Videos were entertaining          -

-

2

20

2

20

3

30

3

30

Questions helped with content     -

-

-

-

2

20

6

60

2

20

Questions were the right level of -

-

1

10

1

10

6

60

2

20

difficulty

Enjoyed challenges              -

-

-

-

5

50

3

30

2

20

Badges encouraged participation -Usability

-

1

10

1

10

6

60

2

20

Easy to navigate                  -

-

-

-

-

-

5

50

5

50

Format made sense             -

-

-

-

-

-

7

70

3

30

Language is easy to understand -

-

1

10

1

10

6

60

2

20

Liked colour scheme             -

-

-

-

3

30

5

50

2

20

Liked fonts                       -

-

-

-

1

10

9

90

-

-

Liked photos                    -

-

-

-

2

20

6

60

2

20

Content is interesting              -

-

1

10

1

10

6

60

2

20

Liked videos                    -

1

10

6

60

3

30


Table 5 Post-intervention outcome satisfaction ratings of the +Connect digital intervention

Question

Very satisfied

Somewhat

Not at all satisfied

n

%

n

%

n

%

Ease of understanding

5

50

5

50

0

0

Look forward being with people

5

50

4

40

1

10

+Connect helped me enjoy life

5

50

4

40

1

10

+Connect helped me feel connected with others

4

40

5

50

1

10

+Connect helped increase social confidence

5

50

4

40

1

10

Helped create new relationships

4

40

3

30

3

30

Helped accept mental health symptoms

3

30

7

70

0

0


of 10 participants agreed that the challenges were enjoyable. See Table 4 for more details.

Seventy percent of participants (7/10) reported that they were somewhat or very satisfied across each outcome criterion assessed. For example, all participants (100%) reported being somewhat satisfied or very satisfied that +Connect was easy to understand and helped them accept their mental health symptoms. Ninety percent of participants (9 out of 10) also reported that +Connect helped them to increase their social confidence, enjoy life, look forward to being with people, and feel more connected with others. However, three out of 10 participants also reported that +Connect did not help create new relationships, which was consistent with the focus on improving the quality of current relationships rather than creating new ones. More details of the outcome satisfaction ratings are shown in Table 5.

Acceptability qualitative outcome data The qualitative findings indicated three key outcomes from engagement with +Connect: improved positive affect (n = 5) improved social interactions (n = 4), and increased social confidence (n = 2). Three out of 10 participants also reported an intent to apply +Connect skills in future social interactions. Participants attributed their positive outcomes of app usage to two main processes induced by the app: learning and self-reflection. These findings were supported by the survey results.

Acceptability qualitative content data Participants reported an overall high level of satisfaction with the app modules; 50-90% of participants found the modules of +Connect to be either helpful or very helpful. In interviews, participants reported Strengths, Gratitude, Sharing Positive News, Three Good Things, and Social Fears as their favourite modules. In terms of content difficulty, participants reported the greatest difficulty was completing the challenges, with five out of 10 participants reporting that they did not do many of the challenges. Participants reported no concerns with the difficulty level of the questions (see Table 4).

There were no differences in participant preferences regarding the type of video (i.e., expert, actor, or shared experience; see Online Resource 2 for more details and examples of representative quotes). Participants generally associated EVs with information provision, AVs with behaviour modelling and fun, and SEVs with motivational examples of how people had successfully applied the skills. Online Resource 2 also describes how the SEVs and AVs were the most well-received videos with all participants stating they were either somewhat or very much satisfied with their usefulness. Most users (90%) also found the SEVs and AVs enjoyable, but while they found the expert videos useful (90%), a smaller number (70%) found it enjoyable.

Usability

All ten participants agreed, or strongly agreed, that the app was easy to navigate and the format made sense (see Online Resource 3 for additional qualitative feedback). Eight out of 10 participants agreed that the language was easy to understand, the content was interesting, and that they liked the photos. Similarly, nine out of 10 participants liked the videos and fonts used in the app. This was supported by the interview feedback (see Table 3). App design feedback primarily focussed on increasing app personalisation, gamification, and feedback functions. A list of participant improvement suggestions is also provided in Online Resource 3.

Safety

There were no adverse events (i.e., psychiatric admissions) recorded during the trial.

Exploratory analyses

Table 2 shows the means and standard deviations across each timepoint. These scores can be used to compute effect sizes that generally suggest acceptability of the intervention; however, we focus on latent trajectory tests to limit subjective interpretation of these patterns. We used a latent trajectory model [66] to estimate the effect size of change in (a) the UCLA-LS and (b) the outcome measures (i.e., SPWB and CDSS) most associated with loneliness during the intervention. As described above, the slope is in terms of standard deviations based on the pre-intervention scores. The mean of both slopes indicated participants were more likely to benefit from the intervention (UCLA-LS: M = - 0.34, SD = 0.24; three measures: M = - 0.29, SD = 0.14). Thus, for both analyses, participants could be expected to have scores that are about 0.3 standard deviations lower at post, and about 0.6 standard deviations lower at follow-up than at pre (i.e., because the slope used was linear). This finding adds to the indications above that the intervention was seen as acceptable.

Discussion

Loneliness in people with psychosis is a significant challenge [1, 2] that is currently neglected within existing psychosocial interventions [3]. While researchers are interested in improving social outcomes and deliver a vast array of psychosocial interventions for those with psychosis, loneliness is not seen as the traditional treatment target. We proposed using a positive psychology approach to promote the development of meaningful relationships and reduce loneliness. As digital tools are increasingly being used for individuals with psychosis [23], we co-designed a smartphone app, called +Connect, that could deliver youth-friendly materials.

Our findings indicate that loneliness may be addressed via digital means using PPI content. Overall, +Connect was found to be feasible and acceptable intervention to address loneliness in young people with psychosis. However, while there is promising evidence that +Connect may mitigate loneliness, further research should consider testing an improved version within a larger trial.

Participants identified Strengths, Gratitude, Sharing Positive News, and Three Good Things as favourite modules. Watching a peer’s experience of doing tasks or sharing their experience within the SEVs was especially important in creating participant engagement with the app, and in encouraging the transfer of skills learnt within the app to real-life context. This is consistent with previous research that suggest that online peer-related material can improve a sense of connection to others with lived experiences [4667, 68]. Furthermore, positive feedback on usability may have been elicited because of consumer involvement in the development phase using groups of young people, where feedback was given on design, gamification, font, photos, content, and videos. Involving consumers within these processes within coproduction design frameworks could plausibly increase the engagement of health services [41, 6970]. While participants reported the EVs as useful, particularly for the provision of information, they did not find the EVs as enjoyable as the other videos. One way forward is to consider using more engaging, fun, and interactive ways to relay academic information, as proposed by one participant (e.g., animations, narrative storytelling, or choose-your-own adventure scenarios).

At least 50% of our participants also met a clinical diagnosis of social anxiety disorder, and this is consistent with studies that have found that individuals with psychosis also report comorbid social anxiety disorder [71, 72]. Furthermore, higher social anxiety symptom severity is associated with higher loneliness, with a known reciprocal relationship between loneliness and social anxiety being evident over a 6-month period [42]. Taken these findings, we proposed that it was crucial to augment interventions targeting loneliness by addressing possible co-occurring social anxiety symptoms. In our case, the content of +Connect was designed to normalize social anxiety.

Study quality and methodological limitations

First, we recruited a small sample size in the first pilot evaluation, in part due to the strict study inclusion and exclusion criteria. For example, we only focused on participants who met a loneliness severity score of above 38 on the UCLA-LS, and excluded substance-induced psychosis, which was a common presenting problem presenting problem at the mental health early psychosis services. We adhered to the study eligibility as our main focus was to get an understanding of the acceptability and feasibility of a proof of concept digital intervention. Hence, although appropriate to our primary aims of developing the intervention for a specific population, our sample size did not allow good statistical power for quantitative tests. Second, at its current phase of development, +Connect is simply a tool to deliver content. While the gamification components attempted to create a sense of achievement through progression (e.g., badges awarded), some participants nonetheless reported a lack of engagement in the early phases of the app. The user may benefit from clearer or multiple indicators of progressions (e.g., adding more functions that signal of a growth of knowledge like representing by building blocks within the home screen in addition to total points won).

Additionally, the app did not entail functionality to facilitate interactions between young people and their peers, or between moderators and young people. Further development can include chatrooms designed for either one or two functions: (1) peer-to-peer interaction, which allows young people to interact with each other including sharing their experiences of doing challenges and having safe opportunities to provide and give social support to each other; (2) moderator-to-user interaction, which allows trained moderators (i.e., clinician or peer moderators) to assist young people to translate skills learnt within the app to real life. The availability of moderator to user chat functions may be especially important because participants reported difficulty with completing challenges. A chatroom dedicated to assisting the translation of app skills to real life will not just provide an opportunity to provide support and to facilitate close monitoring but also allows clinician moderators to provide tailored assistance to the young person. Should such a function exist, it will require both technical and clinical human support to monitor participant safety and any indicators of deteriorating mental state [73].

Future improvements of the app will address participants’ need for personalization, and this may include the capacity to upload participant profile photos and record reflective comments. Participants also verbalized a need to gamify with different rewards schedules. This maybe include either varying types of rewards (e.g., unlocking the ability to track their journey) or delivering random rewards, which may contribute to increased engagement.

Conclusion

We evaluated a pilot smartphone app, called +Connect, in terms of acceptability, feasibility, and usability. We triangulated quantitative and qualitative data to give us a deeper understanding on the feasibility, acceptability, and usability of +Connect. It is likely that +Connect is not just feasible, acceptable, and useable to young people with psychosis, and it holds the promise of mitigating loneliness even at a development phase. A positive psychology approach underpinning the content, as well as the use of engaging shared experience and actor videos, may have led to increased engagement of the program. Further developments are required to make expert videos more interesting and improve the opportunities for participants to interact with peers and, or trained moderators.

Acknowledgements Acknowledgements to Claire Peck, Julia Cheah, Carla McEnery for assistance throughout the development phase, to Amplified Software for digital development, and Ryan O’Hehir for video development. Acknowledgements to staff and consumers of Eastern Health Child and Adolescent Services, Headspace Alfred.

Funding Barbara Dicker Brain Sciences Foundation Grant funding awarded to Lim, Thomas, and Abbott. Funding awarded to Lim, M.H. Higher Education Participation and Partnerships Program for development.

Compliance with ethical standards

Conflict of interest The authors declare no conflict of interests.

References

ciples into practice in mental health contexts. Creative Commons Attribution 30. https://recoverylibrary.unimelb.edu.au/__data/

assets/pdf_file/0010/2659969/Coproduction_putting-principles -into-practice.pdf. Accessed 14 Nov 2018

Affiliations

Michelle H. Lim1,2 • John F. M. Gleeson3 • Thomas L. Rodebaugh4 • Robert Eres1,2 • Katrina M. Long1 • Kit Casey22 23 24 Jo-Anne M. Abbott5 • Neil Thomas23 • David L. Penn3,6

Received: 16 March 2019 | Accepted: 31 January 2020

DOI: 10.1111/eip.12947

ORIGINAL ARTICLE


Wiley


Horyzons USA: A moderated online social intervention for first episode psychosis

Kelsey A. Ludwig25 © | Julia W. Browne1,2 | Arun Nagendra25 | John F. Gleeson3,4 | Simon D'Alfonso4 | David L. Penn1,3 | Mario Alvarez-Jimenez4,5

department of Psychologyand Neuroscience, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 2Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 3School of Behavioural and Health Sciences, Australian Catholic University, Melbourne, Victoria, Australia

4Centre for Youth Mental Health, The University of Melbourne, Parkville, Victoria, Australia

5Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, Victoria, Australia

Correspondence

Kelsey A. Ludwig, The University of North Carolina at Chapel Hill, 235 E. Cameron Ave., Chapel Hill, NC 27599.

Email: kelsey_ludwig@med.unc.edu


Abstract

Aim: We evaluated the feasibility and acceptability of Horyzons, an online social media platform designed to facilitate relationship development among, and introduce therapeutic content to, first-episode psychosis (FEP) clients. We also evaluated whether participation in the platform was related to reduced loneliness, improved social integration and increased psychological well-being.

Methods: Twenty-six participants diagnosed with a schizophrenia spectrum disorder were provided access to the moderated Horyzons platform for 12 weeks. During the intervention period, participants were encouraged to access therapeutic content and social components embedded within the site. Participants were recruited from three first-episode coordinated specialty care clinics in North Carolina and assessed at four time points: baseline, mid-treatment, post-treatment and 1-month follow-up.

Results: Findings indicated that Horyzons was both feasible and very well tolerated, with a 92.3% retention rate and 79.2% of participants actively engaged in the platform. The most commonly identified personal strengths selected by Horyzons users were creativity (61.5%), curiosity (42.3%) and courage (38.5%). Feedback from participants indicated Horyzons could be improved by the development of a smartphone application, expanding the size of the Horyzons community and facilitating private messages between users. Preliminary results with engaged participants showed the greatest improvements in psychosis-related symptoms, followed by self-reported experience of negative emotions, depressive symptoms and loneliness.

Conclusions: This open trial found that Horyzons is both feasible and acceptable to FEP persons early in the course of illness living in the United States.

KEYWORDS

early intervention, first-episode psychosis, loneliness, social integration, social media

recovery include quality of life, perceived social integration and empowerment (Lloyd, King, & Moore, 2010). Social integration has become increasingly central to the conceptualization of recovery and well-being for people experiencing mental health issues, particularly for individuals with psychosis (Delman, Delman, Vezina, & Piselli, 2014).

Despite widespread interest in contributing to the community and desire to feel fully integrated in society, the vast majority (80%) of individuals with psychosis report persistent and impactful experiences of loneliness and social isolation (Badcock et al., 2015; Stain et al., 2012). Research suggests individuals with psychosis are five-to-six times more likely to experience loneliness than persons without a psychiatric condition (Meltzer et al., 2013). Results from the Survey of High Impact Psychosis (SHIP) indicated loneliness and social isolation ranked second on the list of challenges to recovery (Morgan et al., 2017; Lim, Gleeson, Alvarez-Jimenez, & Penn, 2018). Respondents also indicated stigma and fear of social situations prevented community participation among persons with psychosis (Stain et al., 2012), with the majority (69%) avoiding all social activities in the previous year (Morgan et al., 2017).

Deriving less pleasure from and feeling more threatened by inperson social situations may prevent individuals with psychosis from forming new face-to-face relationships or seeking additional support from current contacts (Schneider et al., 2017). Although persons with psychosis may benefit greatly from forming virtual connections with others (Alvarez-Jimenez et al., 2013), commonly used social media platforms may not be appropriate for use with this population. Specifically, intensified social media use may involve certain problematic features including exacerbated symptoms and possible rejection (Torous & Keshavan, 2016). In contrast, internet-based interventions that promote social connection as well as peer and professional supports may be promising tools for decreasing perceived social isolation in this population (Schlosser et al., 2018).

One such social media platform, Horyzons, was developed to promote continued progress toward recovery after discharge from a specialized mental health centre for first-episode psychosis (FEP) in Melbourne, Australia (Alvarez-Jimenez etal., 2013). Horyzons was designed to foster a sense of community, inclusivity and mutual support, which may reduce self-stigma, improve self-esteem and increase self-efficacy, thereby combating feelings of loneliness and promoting social integration (Alvarez-Jimenez et al., 2014). Preliminary findings suggest Horyzons is feasible, safe, acceptable and beneficial for recently discharged FEP clients (Alvarez-Jimenez et al., 2013). Despite the potential for supportive and therapeutic social media platforms to provide cost-effective support for clients transitioning to less specialized care, integrating therapeutic programs like Horyzons (Alvarez-Jimenez et al., 2018) into standard care for FEP is minimally implemented in the United States.

The current study aims to examine the feasibility and acceptability of Horyzons for American clients receiving care at three FEP clinics in North Carolina. We report preliminary results of a small, uncontrolled open trial of Horyzons, including: site usage information, changes in psychological health variables (eg, feelings of loneliness, depressive symptoms) and a summary of participants' feedback regarding the intervention.

2 | METHOD

2.1 | Participants and procedure

Participants were recruited from FEP CSC clinics in North Carolina. Sites included the Outreach and Support Intervention Services (OASIS) in Carrboro, Supporting Hope Opportunities Recovery and Empowerment (SHORE) in Wilmington and Wake Encompass in Raleigh. Each clinic specializes in early identification, individualized recovery and relapse prevention.

Inclusion criteria for participation were (a) ages 18-35; (b) no psychiatric hospitalizations in the last 3 months; (c) meeting DSM-IV criteria for a schizophrenia spectrum disorder; (d) maximum of five lifetime years of treatment with antipsychotic medication; (e) no current suicidal ideation or suicide attempt within the past 2 years; (f) not meeting diagnostic criteria for substance dependence; (g) estimated IQ > 70; (h) Internet access and (i) English language proficiency sufficient to complete assessments. Psychiatric diagnoses were collected from patients' healthcare providers, via chart review and/or through the psychosis, mood and substance use disorder modules from the SCID (First, Spitzer, Gibbon, & Williams, 2002).

Trained raters assessed participants at baseline, mid-treatment (6 weeks), post-treatment (12 weeks) and 1-month follow-up (16 weeks). The project was approved by the UNC-CH Institutional Review Board. Participants provided signed informed consent.

2.2  | Measures

Primary outcome measures included (a) participant use of and satisfaction with Horyzons, which were examined using site usage information (eg, number of logins) and self-report questionnaire (eg, perceived ben-efits/challenges of the intervention); (b) experiences of loneliness examined by the UCLA Loneliness Scale (UCLA; Russell, Peplau, & Ferguson, 1978) and (c) perceived social support and relationship quality measured by the Social Provisions Scale (SPS; Cutrona & Russell, 1987).

Secondary outcome measures included well-being measured by the 18-item Ryff Scales of Psychological Well-Being (PWB; Ryff, 1989), positive and negative emotions assessed by the modified Differential Emotions Scale (mDES; Fredrickson, Tugade, Waugh, & Larkin, 2003) and subjective self-worth measured by the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965).

Exploratory outcome measures included psychosis-related symptoms assessed using the Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, 1987) or Brief Symptom Inventory (BSI; Derogatis, 1993); depressive symptoms examined by the Beck Depression Inventory, Second Edition (BDI-II; Beck, Steer, & Brown, 1996) and social/occupational functioning measured by the First-Episode Social Functioning Scale (FE-SFS; Lecomte et al., 2014).

2.3  | Horyzons

Specific aspects of the Horyzons platform are designed to foster positive social connections among users, including “The Cafe,” where users can post content and comment on other users' posts; “Talk-It-Out” through which users discuss specific issues (eg, handling setbacks), receive support or suggestions and are guided through problem-solving steps; and “Team Up” where users track personal goals (eg, staying fit) and share their progress. Horyzons also integrates therapeutic content from Cognitive Behavioral Therapy (eg, psychoeducation about the interrelatedness of thoughts/feelings/behaviours), Positive Psychology (eg, lessons on self-compassion and gratitude) and Mindfulness and Meditation (eg, mindful walking) that users can complete independently. Psychoeducational materials are divided into 17 “Pathways,” each comprised of a series of related “Steps.” All Pathways and Steps are related to coping (with difficult emotions, etc.), connecting (with others by boosting relationships, etc.) or enhancing (life by promoting happiness). To ensure language and content were applicable to American users, informal Australian expression and slang terms were replaced with equivalents in American English (eg, arvo//afternoon). An additional, optional component of Horyzons involves in-person “Meet-up” events (eg, bowling, playing board games).

Users were also paired with a Horyzons “moderator.” Moderators were Master's level clinical psychology graduate students (n = 5), Licensed Clinical Psychologist (n = 1), and Licensed Clinical Social Worker (n = 1) responsible for tailoring content to users' individual strengths and personal goals. Moderators contacted each user via phone within a week of induction to the platform. The purpose of the initial conversation was to introduce the moderator, explain their role and discuss the user's specific interests, goals and perceived challenges for using the site. For engaged clients, moderators sent personalized messages through the Horyzons platform weekly, which included content or activity suggestions. For inactive clients, moderators followed up with the client via text/call/email weekly or biweekly to discuss and problem-solve barriers (eg, forgotten password). Moderators encouraged client participation on the site through positively reinforcing comments (ie, praise, encouragement and support).

TABLE 1 Participant demographic and clinical characteristics

Characteristic

Active participants (N = 19)

Drop out and inactive participants21 (N = 7)

n

%

n

%

Phase

Cohort 1

10

52.6

2

28.6

Cohort 2

9

47.4

5

71.4

Gender

Male

12

63.2

7

100

Female

7

36.8

0

0

Race

Caucasian

12

63.2

4

57.1

African American

6

31.6

3

42.9

Asian

1

5.3

0

0.0

Ethnicity

Hispanic/Latino

1

5.3

1

14.3

Non-Hispanic/Latino

18

94.7

6

85.7

Diagnosis

Schizophrenia

8

42.1

6

14.3

Schizoaffective

8

42.1

1

85.7

Schizophreniform

1

5.3

0

0.0

Psychosis NOS

2

10.5

0

0.0

Medication type

Atypical

18

94.7

6

85.7

Typical

0

0.0

1

14.3

None

1

5.3

0

0.0

Mean

SD

Mean

SD

Age (years)

25.16

4.05

24.71

2.29

Education (years)

14.68

1.42

13.57

1.62

Maternal education (years)

15.53

2.06

13.71

2.43

Paternal education (years)

15.94

2.92

14.33

2.66

Length FEP program (years)

3.13

2.55

1.71

0.76

PANSS (Trial 1baseline)

Positive total

13.50

2.68

12.00

5.66

Negative total

14.30

4.06

28.00

0.00

General total

31.50

6.49

39.50

7.78

Overall total

59.30

9.36

79.50

13.44

BSI (Trial 2baseline)

47.67

37.27

46.40

31.77


Note: Samples did not significantly differ in any clinical or demographic characteristics outlined above.

Abbreviations: BSI, Brief Symptom Inventory; PANSS, Positive and Negative Syndrome Scale.

individuals who dropped out of the study (n = 2) as well as persons who did not meet the minimum level of engagement in the platform (n = 5).


Moderators conducted daily safety checks, which involved reviewing posts automatically blocked by the system due to inclusion of “risk words” (eg, “death/dead/die/dying”). Any sign of risk (eg, posts about very low mood and suicidal ideation) was followed up by contacting users within 24 hours to provide support and assess risk. Moderators participated in weekly supervision calls with US Principal Investigator (DLP) to discuss client case conceptualization and address client concerns.

2.4 | Statistical analysis

Data analyses were performed using the Statistical Package for the Social Sciences (SPSS, version 24). Statistical significance was defined as P < .05. Descriptive statistics and percentages were used to determine feasibility and acceptability of Horyzons. Standardized changeover-time values were computed to assess potential within-subjects differences. Within-group effect sizes are reported for changes between baseline and mid-treatment, post-test and follow-up. To examine the extent to which different components of Horyzons usage were associated with improvements in outcomes, we computed correlations between Horyzons usage information (eg, number of steps taken) and changes in outcomes between baseline and post-treatment.

3 | RESULTS

3.1 | Participants

Twenty-four participants (92.3%) completed all research assessments. Participants included in the first cohort (n = 12) were recruited from a single clinic (OASIS) and were involved in the project from late 2016 to early 2017. The second cohort included participants from three clinics (OASIS, n = 5; Encompass, n = 3; and SHORE, n = 6) and accessed the platform from early to mid-2018.

Two participants were considered dropouts and were removed from the study due to incarceration or change of housing that precluded assessment completion. Participants endorsed relatively low levels of symptoms at baseline (PANSSTotal: MC1 = 62.67, SDC1 = 12.24; BSITotal: MC2 = 47.21, SDC2 = 34.14). Participants in

TABLE 2 Horyzons acceptability and feasibility

Horyzons component M (SD)

All, n = 24

Active participants, n = 19

Inactive participants21, n = 5

Horyzons usage information11

Logins

32.88 (31.84)

40.00 (32.19)

5.80 (2.77)

Suggestions completed (%)

35.47 (33.30)

43.06 (32.81)

6.67 (14.91)

Actions

12.29 (46.31)

15.47 (51.86)

0.20 (0.45)

Comments

7.29 (12.07)

9.11 (13.01)

0.40 (0.89)

Talking points

1.08 (2.47)

1.37 (2.71)

0.00 (0.00)

Talk-it-outs

2.83 (4.37)

3.58 (4.65)

0.00 (0.00)

Steps

5.46 (6.98)

6.89 (7.19)

0.00 (0.00)

Posts

6.08 (9.79)

7.53 (10.57)

0.60 (0.89)

Total itemsb

35.04 (61.20)

43.95 (66.20)

1.20 (1.64)

Post-treatment feedback

Total measure M (SD)

3.78 (1.07)

3.88 (1.07)

3.43 (0.96)

How easy was it to use HORYZONS?

3.96 (0.86)

4.11 (0.86)

3.40 (0.55)

How much did you enjoy HORYZONS?

3.54 (1.10)

3.63 (1.16)

3.20 (0.84)

How helpful was HORYZONS for you?

3.85 (0.90)

4.00 (.94)

3.40 (1.34)

How safe did you feel using HORYZONS?

4.17(1.09)

4.37 (0.96)

3.40 (1.34)

How would you rate the quality of social interactions you had in the cafe?

3.67 (1.24)

3.79 (1.18)

3.20 (1.48)

How much did HORYZONS help you look forward to being with people?

3.50(1.25)

3.37 (1.30)

4.00 (1.00)

aPersons who did not meet the minimum level of engagement in the platform only (n = 5); these analyses did not include study dropouts (n = 2). bTotal items refers to the number of site activities completed by participants (ie, sum of actions, comments, talking points, talk-it-outs, steps and posts).

TABLE 3 Within-subjects change in outcome variables (n = 19)a

Measure (visit)

M

SD

BL - MT

BL - PT

BL - FU

Primary

Effect sizes (d)

UCLA (BL)

28.74

17.00

0.27

-0.01

-0.05

UCLA (MT)

24.21

15.68

UCLA (PT)

28.84

15.66

UCLA(FU)

29.63

17.22

SPS (BL)

65.84

9.26

0.03

0.10

-0.18

SPS (MT)

66.16

9.26

SPS (PT)

66.79

10.15

SPS(FU)

64.21

6.29

Secondary

PWB (BL)

73.53

16.84

0.13

0.11

0.07

PWB (MT)

75.74

12.08

PWB (PT)

75.37

10.40

PWB (FU)

74.70

15.68

mDES Pos (BL)

26.63

9.08

0.06

-0.03

0.12

mDES Pos (MT)

27.21

7.35

mDES Pos (PT)

26.32

7.25

mDES Pos (FU)

27.74

7.94

mDES Neg (BL)

12.21

6.71

0.27

-0.19

0.27

mDES Neg (MT)

10.37

7.42

mDES Neg (PT)

13.47

8.64

mDES Neg(FU)

10.37

7.77

RSES (BL)

29.58

6.85

0.07

-0.02

-0.15

RSES (MT)

30.05

5.55

RSES (PT)

29.47

5.55

RSES(FU)

28.53

7.49

Exploratory

PANSS total (BL), n = 10

59.30

9.36

-

0.81

0.65

PANSS total (PT), n = 10

51.70

7.67

PANSS total (FU), n = 10

53.20

6.91

BSI total (BL), n = 9

47.67

37.27

0.19

-0.01

0.08

BSI total (MT), n = 9

40.44

28.42

BSI total (PT), n = 9

48.22

29.36

BSI total (FU), n = 9

44.67

21.75

BDI (BL)

13.84

11.33

0.30

0.04

0.14

BDI (MT)

10.47

9.44

BDI (PT)

13.37

10.01

BDI (FU)

12.21

10.02

FE-SFS abilityb (BL)

3.22

0.40

-

0.05

-

FE-SFS abilityb (PT)

3.24

0.37

FE-SFS behaviourb (BL)

2.93

0.41

-

0.18

-

FE-SFS behaviourb (PT)

3.00

0.38

Notes: All Cohen's d values represent magnitude of the change based on standard deviations from baseline. Positive effect sizes reflect improvements whereas negative effect sizes indicate deterioration.

Abbreviations: UCLA, UCLA Loneliness Scale; SPS, Social Provisions Scale; PWB, Ryff Scales of Psychological Well-being; mDES Pos/Neg, modified Differential Emotions Scale, Positive/Negative Subscales; RSES, Rosenberg Self-Esteem Scale; PANSS, Positive and Negative Syndrome Scale; BSI, Brief Symptom Inventory; BDI, Beck Depression Inventory; FE-SFS, First Episode Social Functioning Scale.

aActive participants only included in above analyses. bComposite score.

cohort 1 endorsed less social support than individuals in cohort 2 (SPS: MC1 = 61.90, SDC1 = 3.48; MC2 = 70.14, SDC2 = 9.84; t (22) = 2.52, P = .02). Cohorts did not significantly differ on any other demographic, clinical or outcome variables at baseline.

We defined minimal platform usage as an average of at least one login per week (12 total logins) and at least 10 instances of site utilization (eg, comments, talking points, etc.). Active participants (n = 19) reached or surpassed this standard, whereas Inactive participants (n = 5) did not reach minimum usage. At baseline, active participants endorsed less social support (d = -0.54) and increased positive (d = 0.48) and negative affect (d = 0.36) than inactive participants. Demographic and clinical characteristics of the sample are presented (Table 1).

3.2 | Feasibility and acceptability

Participants logged into Horyzons an average of 32.9 times (SD = 31.84; range: 3-134) over the course of treatment. Most participants found the site easy to use, helpful and safe. Inactive participants were generally less satisfied with Horyzons than active participants. However, inactive participants described Horyzons as more helpful in terms of looking forward to being with people (Table 2).

Written feedback suggested the most well-received aspects of the site were positive interactions with other users and the sense of community. Suggestions for improvement included creating an app accessible via smartphone, expanding the platform to include additional users and facilitating private messages. Additional usage information and feedback about Horyzons are provided (Table 2).

As identifying and promoting strengths is a core component of Horyzons, clients were asked to identify areas of strength they found personally relevant and meaningful during induction to the Horyzons platform. The most commonly identified strengths selected by participants were creativity (61.5%), curiosity (42.3%) and courage (38.5%). The least commonly identified strengths were self-control, social intelligence, teamwork and leadership (all 7.7%).

Steps completed by participants were most often acceptancebased or related to mindfulness and meditation. The most common steps taken were mindful thoughts and anchor yourself (both completed 10 times total), followed by being with difficulty, body and breath, and body scan (taken 7 times each). The most common actions (activities designed to reinforce strengths or practice new skills) completed were related to improving emotional experiences and preparing for jobs, including: being with difficult emotions (completed 12 times), body scan (completed 9 times), nailing the interview (completed 8 times) and how to write a resume and getting your public persona ready (each completed 7 times).

TABLE 4 Relationships between usage and changes in outcomes (n = 19)a

Measure

Number of logins

% Suggestions completed

Actions

Steps

Social Provisions Scale

-0.33

-0.39

-0.20

-0.26

UCLA Loneliness

-0.41

-0.05

-0.02

0.01

Psychological well-being

0.64**

0.17

-0.13

0.09

mDES positive

0.44

0.03

0.19

0.22

mDES negative

-0.26

-0.05

-0.16

-0.28

Rosenberg Self-esteem Scale

0.29

-0.05

-0.05

0.04

PANSS total score, N = 10

-0.50

0.47

0.53

0.41

BSI total score, N = 9

0.47

0.53

0.39

0.28

Beck depression inventory

-0.34

-0.05

0.15

0.02

FE-SFS ability composite

0.32

-0.80

-0.19

-0.22

FE-SFS behaviour composite

0.34

0.03

-0.23

-0.27

Measure

Posts

Comments

TIO

Talking points

Social provisions scale

-0.14

-0.14

-0.29

-0.21

UCLA Loneliness

-0.31

-0.45*

-0.34

-0.31

Psychological well-being

0.61**

0 72***

0.57**

0.34

mDES positive

0.35

0.57**

0.46*

0.55*

mDES negative

-0.54*

-0.69***

-0.53*

-0.55*

Rosenberg Self-esteem Scale

0.38

0.43

0.29

0.26

PANSS total score, N = 10

-0.41

-0.49

-0.36

0.05

BSI total score, N = 9

-0.21

0.31

0.47

-0.08

Beck depression inventory

-0.52*

-0.62**

-0.53*

-0.34

FE-SFS ability composite

0.28

0.31

0.32

0.13

FE-SFS behaviour composite

0.62**

0.38

0.21

-0.01

Notes: Primary Outcomes: UCLA Loneliness Scale (UCLA), Social Provisions Scale (SPS); Secondary Outcomes: Ryff Scales of Psychological Well-being (PWB), modified Differential Emotions Scale, Positive/ Negative Subscales (mDES Pos/Neg), Rosenberg Self-Esteem Scale (RSES); Exploratory Outcomes: Positive and Negative Syndrome Scale (PANSS), Brief Symptom Inventory (BSI), Beck Depression Inventory (BDI), First Episode Social Functioning Scale (FE-SFS).

aActive participants only included in above analyses.

*P < .05.; **P < .01.; ***P < .001.


3.3  | Changes in outcomes

Five users were removed from subsequent analyses as they did not meet the minimum level of engagement. Thus, following analyses include engaged participants only. Regarding primary outcomes, reports of loneliness showed the largest improvement from baseline to mid-treatment (Table 3). Changes in participants' perceived social support and relationship quality were in the expected direction from baseline to mid-treatment and post-treatment, although modest and not maintained at follow-up (Table 3).

Negative emotions showed the greatest reductions with moderate changes between baseline and mid-treatment/follow-up. Participants' endorsement of negative emotions demonstrated small increases from baseline to post-treatment (Table 3). Involvement in Horyzons did not significantly impact the secondary outcomes of psychological well-being, positive emotions or self-esteem (Table 3).

Exploratory outcomes showed the strongest effect at post-treatment, with greatest improvements in psychosis-related symptoms. Small-to-medium effect size improvements in depressive symptoms were observed from baseline to post-treatment/follow-up. Finally, participants' self-reported social functioning indicated slight improvements (Table 3). Post hoc tests revealed the living skills ability and behaviour subscales evidenced the greatest improvement from baseline to post-treatment. Improvements were generally maintained but attenuated at follow-up.

3.4  | Effect of Horyzons usage on outcomes

Posting on the Cafe, commenting on others' posts and discussing an issue through the Talk-It-Out feature showed medium-to-high correlations with increases in psychological well-being and positive emotions as well as reductions in depressive symptoms and negative emotions (Table 4). Login frequency was significantly associated with improvements in psychological well-being for actively engaged participants. Actions completed, suggestions followed and steps taken were not significantly related to changes in outcomes (Table 4).

4 | DISCUSSION

This study provides preliminary evidence that Horyzons is a feasible and acceptable intervention for individuals with FEP in the United States. The overall retention rate across both cohorts (92.3%) indicated the intervention was well-tolerated. This finding was supported by participants' overall engagement and generally positive feedback about Horyzons. Preliminary results showed the greatest improvements in psychosis-related symptoms, followed by negative emotions, depressive symptoms and loneliness. Preliminary findings suggest active engagement in Horyzons was associated with enhanced social integration, improved psychological well-being, increased positive emotions, as well as decreased negative emotions and depressive symptoms.

To our knowledge, this is the first online, strengths-based, social networking intervention to have been successfully implemented with FEP in the United States. Work by Schlosser et al. (2018) recently demonstrated the feasibility and acceptability of PRIME, an online therapy intervention delivered via mobile app. PRIME was designed to target impaired motivation through goal-setting, achievement tracking and individualized coaching. Key features that distinguish Horyzons from the few extant online interventions for FEP include its emphasis on characterological strengths, integration in coordinated specialty care settings and use of a community of peers to reduce loneliness and improve social integration.

Emphasizing strengths may provide the kind of support and encouragement needed for young persons with psychosis to better cope with symptoms and make progress toward personally relevant goals (Browne et al., 2018). The breakdown of strengths selected by individuals in the current study corresponds well with endorsements from previous samples of FEP participants (Browne et al., 2018) and normative groups (Seligman, Steen, Park, & Peterson, 2005). The power of enhancing strengths was also evident in feedback received from Horyzons users. As one active user noted, “I found the Talk-It-Out section and the Cafe most helpful because they helped me gain clarity on who I am and what I stand for.”

Notably, moderate reductions in experiences of loneliness, depressive symptoms and negative emotions were demonstrated after only 6 weeks of platform usage. As research suggests psychological well-being is closely associated with mental health recovery in FEP (Browne et al., 2017), the fact that the number of logins and social networking components of Horyzons were related to improved psychological well-being is striking. Although we cannot draw firm conclusions about the mechanism of psychological change brought on by Horyzons at present, this finding suggests mere exposure to the site may provide benefits even in the absence of engagement with therapeutic content (eg, steps/pathways) or prompted behaviour change (eg, actions). The current iteration of Horyzons precludes accurate recording of the frequency with which clients complete actions. Changes in clinical outcome variables may also be particularly encouraging considering this study recruited only stable outpatients currently receiving services at specialty care clinics.

Our findings also suggest different ways of engaging with the platform seem to be associated with improvements in certain outcome variables such as loneliness and depressive symptoms. It could be that active users who were self-directed and navigated the site independently and according to their preferences experienced Horyzons as supporting their innate needs for autonomy, competence and relatedness (Ryan & Deci, 2000). It is also possible that users' decision to utilize social networking features of the site, such as posting and commenting on the cafe or discussing issues and receiving support in a Talk-It-Out, may have been key to facilitating changes in outcomes. As individuals with psychosis tend to feel less comfortable and more threatened in the presence of others (Schneider et al., 2017), Horyzons may provide a sense of safety and community that values inclusivity, non-judgement and support that may differ from other forms of social contact. Taken together, Horyzons, like most treatments, is not a one-size fits-all intervention.

Limitations of the current study include a small sample size and lack of a control condition. The correlational nature of this research also precludes our ability to infer causation about any observed changes in outcomes. Additionally, the short duration of this study as well as the relatively brief follow-up period prohibit our ability to draw firm conclusions about the reliability and sustainability of relationships between Horyzons usage and outcomes. As such, these findings should be considered preliminary. Moreover, assessments relied heavily on self-report questionnaires, which can be greatly impacted by recall bias and/or respondents' current emotional states (Michalska da Rocha, Rhodes, Vasilopoulou, & Hutton, 2018). Finally, the present findings should be interpreted with thoughtful consideration as outcome analyses included only individuals who reached a predetermined level of engagement.

Despite these limitations, access to a moderated and strengths-based social media platform such as Horyzons may provide unique treatment benefits and serve as a supportive adjunct to care for clients currently engaged in FEP treatment. Identifying individual characteristics and contexts that indicate which persons may especially need or benefit from this type of intervention merits further investigation. Future research should consider evaluating Horyzons in the context of a randomized controlled trial with the inclusion of a comparison group, which is currently underway at Orygen Youth Health in Melbourne, Australia (Alvarez-Jimenez et al., 2018).

ORCID

Kelsey A. Ludwig https://orcid.org/0000-0002-3687-4725

REFERENCES

Alvarez-Jimenez, M., Bendall, S., Koval, P., Rice, S., Cagliarini, D.....

Gleeson, J. F. (2018). HORYZONS trial: Protocol for a randomised controlled trial of a moderated online social therapy to maintain treatment effects from first-episode psychosis services. British Medical Journal, 9, e024104.

Alvarez-Jimenez, M., Bendall, S., Lederman, R., Wadley, G., Chinnery, G., Vargas, S.....Gleeson, J. (2013). On the HORYZON: Moderated online

social therapy for long-term recovery in first episode psychosis. Schizophrenia Research, 143(1), 143-149.

Alvarez-Jimenez, M., Alcazar-Corcoles, M. A., Gonzalez-Blanch, C., Bendall, S., McGorry, P. D., & Gleeson, J. F. (2014). Online, social media and mobile technologies for psychosis treatment: A systematic review on novel user-led interventions. Schizophrenia Research, 156(1), 96-106. Badcock, J. C., Shah, S., Mackinnon, A., Stain, H. J., Galletly, C., Jablensky, A., & Morgan, V. A. (2015). Loneliness in psychotic disorders and its association with cognitive function and symptom profile. Schizophrenia Research, 169(1-3), 268-273.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck depression inventory-II. San Antonio, TX: Psychological Corporation.

Browne, J., Estroff, S. E., Ludwig, K. A., Merritt, C., Meyer-Kalos, P. S., Mueser, K. T.....Penn, D. L. (2018). Character strengths of individuals

with first episode psychosis in individual resiliency training. Schizophrenia Research, 195, 448-454.

Browne, J., Penn, D. L., Meyer-Kalos, P. S., Mueser, K. T., Estroff, S. E., Brunette, M. F..... Kane, J. M. (2017). Psychological well-being and

mental health recovery in the NIMH RAISE early treatment program. Schizophrenia Research, 185, 167-172.

Cutrona, C. E., & Russell, D. (1987). The provisions of social relationships and adaptation to stress. In W. H. Jones & D. Perlman (Eds.), Advances in personal relationships (Vol. 1, pp. 37-67). Greenwich, CT: JAI Press.

Delman, J., Delman, D. R., Vezina, B. R., & Piselli, J. (2014). Peer led recovery learning communities: Expanding social integration opportunities for people with lived experience of psychiatric disability and emotional distress. Global Journal of Community Psychology Practice, 5(1), 1-11.

Derogatis, L. R. (1993). The brief symptom inventory (BSI): Administration, scoring and procedures manual. Minneapolis, MN: National Computer Systems.

First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (2002). Structured clinical interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P). New York, NY: John Wiley & Sons.

Fredrickson, B.,Tugade, M. M., Waugh, C. E., & Larkin, G. R. (2003). What good are positive emotions in crises? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001. Journal of Personality and Social Psychology, 84(2), 365-376.

Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13 (2), 261-276.

Lecomte, T., Corbiere, M., Ehmann, T., Addington, J., Abdel-Baki, A., & Macewan, B. (2014). Development and preliminary validation of the first episode social functioning scale for early psychosis. Psychiatry Research, 216, 412-417.

Lim, M. H., Gleeson, J. F. M., Alvarez-Jimenez, M., & Penn, D. L. (2018). Loneliness in psychosis: A systematic review. Social Psychiatry and Psychiatric Epidemiology, 53(3), 221-238.

Lloyd, C., King, R., & Moore, L. (2010). Subjective and objective indicators of recovery in severe mental illness: A cross-sectional study. The International Journal of Social Psychiatry, 56(3), 220-229.

Meltzer, H., Bebbington, P., Dennis, M. S., Jenkins, R., McManus, S., & Brugha, T. S. (2013). Feelings of loneliness among adults with mental disorder. Social Psychiatry and Psychiatric Epidemiology, 48(1), 5-13.

Michalska da Rocha, B., Rhodes, S., Vasilopoulou, E., & Hutton, P. (2018). Loneliness in psychosis: A meta-analytical review. Schizophrenia Bulletin, 44(1), 114-125.

Morgan, V. A., Waterreus, A., Carr, V., Castle, D., Cohen, M., Harvey, C.....

Neil, A. L. (2017). Responding to challenges for people with psychotic illness: Updated evidence from the survey of high impact psychosis. Australian & New Zealand Journal of Psychiatry, 51(2), 124-140.

Roe, D., Mashiach-Eizenberg, M., & Lysaker, P. H. (2011). The relation between objective and subjective domains of recovery among persons with schizophrenia-related disorders. Schizophrenia Research, 131(1), 133-138.

Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

Russell, D., Peplau, L. A., & Ferguson, M. L. (1978). Developing a measure of loneliness. Journal of Personality Assessment, 42, 290-294.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78.

Ryff, C. D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57(6), 1069-1081.

Schlosser, D. A., Campellone, T. R., Truong, B., Etter, K., Vergani, S., Komaiko, K., & Vinogradov, S. (2018). Efficacy of PRIME: A Mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophrenia Bulletin, 44(5), 1010-1020.

Schneider, M., Reininghaus, U., van Nierop, M., Janssens, M., Myin-Germeys, I., & GROUP Investigators. (2017). Does the social functioning scale reflect real-life social functioning? An experience sampling study in patients with a non-affective psychotic disorder and healthy control individuals. Psychological   Medicine, 47(16),

2777-2786.

Seligman, M. E. P., Steen, T. A., Park, N., & Peterson, C. (2005). Positive psychology progress: Empirical validation of interventions. American Psychologist, 60(5), 410-421.

Stain, H. J., Galletly, C. A., Clark, S., Wilson, J., Killen, E. A., Anthes, L.....

Harvey, C. (2012). Understanding the social costs of psychosis: The experience of adults affected by psychosis identified within second Australian national survey of psychosis. Australian & New Zealand Journal of Psychiatry, 46(9), 879-889.

Torous, J., & Keshavan, M. (2016). The role of social media in schizophrenia: Evaluating risks, benefits, and potential. Current Opinion in Psychiatry, 29(3), 190-195.

How to cite this article: Ludwig KA, Browne JW, Nagendra A, et al. Horyzons USA: A moderated online social intervention for first episode psychosis. Early Intervention in Psychiatry. 2020;1-9. https://doi.org/10.1111/eip.12947

In the public domain

ISSN: 0022-006X


2020, Vol. 88, No. 10, 923-936

http://dx.doi.org/10.1037/ccp0000599

Mobile Enhancement of Motivation in Schizophrenia: A Pilot Randomized Controlled Trial of a Personalized Text Message Intervention for Motivation Deficits

Lauren Luther

Massachusetts General Hospital, Boston, Massachusetts, and Indiana University-Purdue University Indianapolis

Richard Holden

Indiana University School of Medicine and Regenstrief Institute, Indianapolis, Indiana

Bryan McCormick

Temple University

Melanie W. Fischer, Annalee V. Johnson-Kwochka, and Kyle S. Minor

Indiana University-Purdue University Indianapolis

Chris L. Lapish

Indiana University-Purdue University Indianapolis

Michelle P. Salyers

Indiana University-Purdue University Indianapolis

Objective: Motivation deficits remain an unmet treatment need in schizophrenia. Recent research has identified mechanisms underlying motivation deficits (i.e., impaired effort-cost computations, reduced future reward-value representation maintenance) that may be effective treatment targets to improve motivation. This study tested the feasibility and preliminary effectiveness of Mobile Enhancement of Motivation in Schizophrenia (MEMS), an intervention that leverages mobile technology to target these mechanisms with text messages. Method: Fifty-six participants with a schizophrenia-spectrum disorder were randomized to MEMS (n = 27) or a control condition (n = 29). All participants set recovery goals to complete over 8 weeks. Participants in the MEMS group additionally received personalized, interactive text messages on their personal cellphones each weekday. Results: Retention and engagement in MEMS were high: 92.6% completed 8 weeks of MEMS, with an 86.1% text message response rate, and 100% reported being satisfied with the text messages. Compared to participants in the control condition, the participants in the MEMS condition had significantly greater improvements in interviewer-rated motivation and anticipatory pleasure and attained significantly more recovery-oriented goals at 8 weeks. There were no significant group differences in purported mechanisms (performance-based effort-cost computations and future reward-value representations) or in self-reported motivation, quality of life, or functioning. Conclusion: Results demonstrate that MEMS is feasible as a brief, low-intensity mobile intervention that could effectively improve some aspects of motivation (i.e., initiation and maintenance of goal-directed behaviors) and recovery goal attainment for those with schizophrenia-spectrum disorders. More work is needed with larger samples and to understand the mechanisms of change in MEMS.

This article was published Online First August 13, 2020.

Lauren Luther, Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, and Department of Psychology, Indiana University-Purdue University Indianapolis; Melanie W. Fischer, © Annalee V. Johnson-Kwochka, and Kyle S. Minor, Department of Psychology, Indiana University-Purdue University Indianapolis; Richard Holden, Department of Medicine, Indiana University School of Medicine, and Indiana University Center for Aging Research, Regenstrief Institute, Indianapolis, Indiana; © Chris L. Lapish, Department of Psychology, Indiana University-Purdue University Indianapolis; © Bryan McCormick, Department of Health and Rehabilitation Sciences in the College of Public Health, Temple University; Michelle P. Salyers, Department of Psychology, Indiana University-Purdue University Indianapolis.

The outcome results from the clinical trial have not been previously published. There is one paper that has been published from the baseline data from this trial; this paper looked at the relationship between self-reported and clinician-rated motivation and looked at moderators (e.g., metacognition and neurocognition) of this relationship.

Preliminary results from this work were presented at the 32nd annual meeting of the Society for Research in Psychopathology and the 2018 Translational Science Conference, and the 6th Biennial Schizophrenia International Research Society Conference. This trial was preregistered on ClinicalTrials.gov (NCT03059771). This work was supported by the William and Dorothy Bevan Scholarship from the American Psychological Foundation, the Indiana Clinical and Translational Sciences Institute (ICTSI) and the ICTSI Clinical Research Center (Grant UL1TR001108), and the National Institutes of Mental Health (Grant T32MH016259). The authors declare no conflicts of interest and are incredibly grateful to the participants who made this work possible.

Correspondence concerning this article should be addressed to Lauren Luther, Department of Psychiatry, Massachusetts General Hospital, 149 13th Street, Room 2603, Charlestown, MA 02129. E-mail: lluther1@mgh.harvard.edu

What is the public health significance of this article?

This study suggests that mobile text message interventions may improve motivation and help people with schizophrenia attain their personalized life goals. Further, this study suggests that text message interventions delivered via personal cell phones are feasible in those with schizophrenia.

Keywords: mhealth, negative symptoms, psychosis

Supplemental materials: http://dx.doi.org/10.1037/ccp0000599.supp

Schizophrenia is a severe mental illness (SMI) accounting for over 155 billion dollars in yearly treatment costs and lost wages in the United States alone (Cloutier et al., 2016). Research suggests that motivation deficits are a key factor affecting functional disability and quality of life (Fervaha, Agid, Takeuchi, Foussias, & Remington, 2013; Fervaha, Foussias, Agid, & Remington, 2014) and are barriers to attaining life goals that can facilitate recovery for people with schizophrenia (Clarke, Oades, Crowe, Caputi, & Deane, 2009). Yet motivation deficits remain an unmet clinical need, as most pharmacological and psychosocial interventions have demonstrated limited efficacy in ameliorating these symptoms (Fusar-Poli et al., 2015).

Mobile interventions are a promising way to more effectively treat these deficits. In particular, text message interventions may be uniquely promising and feasible among people with schizophrenia since text messages are already commonly used among many with schizophrenia (Ben-Zeev, Davis, Kaiser, Krzsos, & Drake, 2013; Naslund, Aschbrenner, & Bartels, 2016), may be less technologically and cognitively challenging to use than some mobile application-based interventions, and may be more accessible than mobile application-based interventions as they do not require a specific type of phone (e.g., smart phone), mobile phone operating system, or costly cellular data. Further, extant studies have found that mobile text message interventions are feasible, acceptable, and clinically promising tools to support a range of outcomes in schizophrenia. Participants receiving daily text messages generally report high levels of satisfaction and usability with text messages (Ben-Zeev, Kaiser, & Krzos, 2014; Granholm, Ben-Zeev, Link, Bradshaw, & Holden, 2012; Kannisto, Adams, Koivunen, Kata-jisto, & Valimaki, 2015). Participant retention and text message response rates in mobile intervention studies have been high (Ben-Zeev, Kaiser, et al., 2014; Granholm et al., 2012; Pijnenborg et al., 2010), supporting the feasibility of this approach. Moreover, initial studies have demonstrated that text-messaging interventions are potentially effective in improving a range of targeted domains, including medication adherence, positive symptoms, and social functioning in schizophrenia (Granholm et al., 2012; Montes, Medina, Gomez-Beneyto, & Maurino, 2012; Pijnenborg et al., 2010; Spaniel et al., 2008).

Despite these promising results, few mobile interventions have specifically targeted negative symptoms such as reduced motivation. Mobile text-messaging interventions may be particularly useful for improving motivation, given that they can help to cue, sequence, and reinforce goal-directed behavior in real time in a person’s daily environment. For example, in contrast to officebased treatment where steps and barriers toward achieving a goal are discussed outside of the context where they will be implemented, mobile text message interventions can provide personalized in the moment interventions in the actual environments where participants are trying to implement strategies or perform steps that facilitate goal achievement. Given these possibilities, this study used mobile technology to deliver a motivation intervention via text messages in real-time, real-world settings. Further, since prior work has found that technology-based interventions targeting potential underlying mechanisms may yield greater effects (Webb, Joseph, Yardley, & Michie, 2010), we aimed to improve motivation by targeting two impaired reward-processing mechanisms posited to underlie motivation deficits in schizophrenia: (a) effortcost computations, and (b) maintenance of reward-value representations (Gold, Waltz, Prentice, Morris, & Heerey, 2008; Strauss, Waltz, & Gold, 2014). Broadly, effort-cost computations involve generating representations of the perceived effort (or cost such as energy, time) and rewards linked to completing a task/goal, including the magnitude of the reward and probability of reward receipt (Green, Horan, Barch, & Gold, 2015), and then integrating this information to evaluate whether the reward is worth the effort (Strauss et al., 2014). Drawing from tasks initially used in preclin-ical animal models (Salamone, Cousins, McCullough, Carriero, & Berkowitz, 1994), researchers have primarily assessed effort-cost computations using the effort expenditure for rewards task (EEfRT; Treadway, Buckholtz, Schwartzman, Lambert, & Zald, 2009) wherein participants choose between completing an easy effort task that provides low monetary rewards or a relatively harder effort task that provides greater monetary rewards. In addition, the probability of receiving the monetary rewards if the chosen task is successfully completed varies across trials. On this task, compared to controls, schizophrenia participants are less likely to choose the hard effort option when the rewards and probability of reward receipt are the highest (Barch, Treadway, & Schoen, 2014; Fervaha et al., 2013; McCarthy, Treadway, Bennett, & Blanchard, 2016; Reddy et al., 2015; Treadway, Peterman, Zald, & Park, 2015) but select about the same amount (Barch et al., 2014; Reddy et al., 2015) or even more (Fervaha et al., 2013; McCarthy et al., 2016) hard effort options than controls on trials with lower reward receipt probability and magnitude. In other words, people with schizophrenia allocate less effort on maximally rewarding tasks than controls. Several studies have found that greater motivation deficits or negative symptoms are associated with choosing fewer hard EEfRT tasks, particularly in the high reward, high probability conditions (Barch et al., 2014; Fervaha et al., 2013; Horan et al., 2015). Taken together, motivational deficits may reflect difficulty integrating information about the cost (i.e., effort) and reward (e.g., magnitude, probability) of a task to identify when it is most advantageous to allocate effort (McCarthy et al., 2016; Treadway et al., 2015).

Additional work in schizophrenia demonstrates that motivational deficits are linked to difficulties maintaining (i.e., “hold[ing] in mind”; Gard et al., 2011) mental representations of the value of future rewards (Gold et al., 2008; Strauss et al., 2014). Thus, temporally distant rewards may be poorly represented and undervalued, especially compared to more immediate rewards (Heerey, Matveeva, & Gold, 2011). Indeed, many with schizophrenia have difficulty sustaining effort for long-term goals or vocational or educational programs (Harding et al., 2008; Kurtz, Rose, & Wexler, 2011), especially when the rewards are temporally distant (e.g., paycheck, degree). Using a delay discounting task, several studies have found that schizophrenia participants discount the value of future rewards more steeply than controls (Ahn et al., 2011; Heerey et al., 2011; Heerey, Robinson, McMahon, & Gold, 2007; L. Q. Yu et al., 2017), suggesting they have greater difficulty representing and thus devalue future rewards. Notably, Heerey et al. (2011) found that greater difficulty representing future rewards was related to reduced motivation. Others have found that value maintenance even over a brief time is impaired and associated with reduced motivation in schizophrenia (Gard et al., 2011). Thus, motivational impairments in schizophrenia may also stem from difficulties identifying and maintaining reward-value representations needed to guide long-term goal-directed behavior.

Although these results provide converging evidence that impaired effort-cost computations and future reward-value representations are mechanisms underlying reduced motivated behavior in schizophrenia, little work has been done to translate these findings into novel motivation targets and treatments. This study tested an intervention that leverages mobile technology to target these two mechanisms in real-world settings. Mobile text-messaging interventions may be particularly apt at targeting these mechanisms for several reasons. First, effort-cost computations are made throughout a person’s daily life (e.g., making favorite meal from scratch vs. making a less flavorful frozen meal version), and mobile text message interventions can provide real-time services to support adaptative effort-cost computations. Mobile text message interventions can guide effective effort allocation by cueing and reinforcing engagement in high-effort, high-reward tasks (e.g., looking/ applying for jobs involving animals) that are important to long term-goals (e.g., becoming a veterinary technician). Mobile text message interventions can also deliver frequent reminders to promote and maintain reward-value maintenance to guide behavior to support long-term goal attainment (Strauss et al., 2014).

To date, we are aware of only one mobile intervention that has targeted motivation. Schlosser et al. (2018) used a mobile app-based intervention for people with early psychosis and found that the 12-week intervention led to significant improvements in social motivation components (i.e., anticipated pleasure and effort for social tasks); trend improvements in self-reported motivation/plea-sure symptoms; and no significant changes in clinician-rated negative symptoms, functioning, or quality of life. However, compared to mobile app-based interventions, mobile text message interventions may be more effective at targeting motivation for those across the schizophrenia-spectrum given that text message interventions may have fewer barriers to treatment initiation and engagement (i.e., does not require a smart phone or cellular data) and may be more accessible and impactful for those with a range of mobile phone experience and cognitive abilities. Yet, to date, mobile interventions targeting other symptoms/domains have found limited effectiveness for improving motivation or other negative symptoms (Ben-Zeev, Brenner, et al., 2014; Granholm et al., 2012). Further, mobile intervention, particularly text message studies, generally have yet to move beyond feasibility studies, resulting in calls for more rigorous randomized designs (Naslund, Marsch, McHugo, & Bartels, 2015; Payne, Lister, West, & Bernhardt, 2015). An additional limitation is that many extant studies provide participants cellphones only for the study period, reducing the results’ ecological validity and clinical utility.

To address these gaps, this pilot study tests the feasibility and preliminary effectiveness of a mobile text-messaging intervention, Mobile Enhancement of Motivation in Schizophrenia (MEMS), which targets effort-cost computations and future reward-value representation maintenance to improve motivation. We used a randomized design to identify whether MEMS would lead to improvements in outcomes above the effects of a group who only engaged in a goal-setting session—a common method to target motivation (Clarke, Oades, Crowe, & Deane, 2006). We hypothesized that MEMS would lead to greater improvements in our main therapeutic targets (i.e., primary outcomes) of effort-cost computations, future reward-value representations, interviewerrated and self-reported motivation, and overall goal attainment compared to goal-setting alone. We also explored whether there were group differences in the more distal outcomes (i.e., secondary outcomes) of quality of life, functioning, neurocognition, and additional symptoms. Finally, we tested the feasibility of using personal mobile phones to deliver MEMS, including examining MEMS engagement, usability, and satisfaction.

Method

Participants

Participants were recruited from a community mental health center serving outpatients with SMI. Participants were eligible if they: (a) were >18 years old, (b) had a Structured Clinical Interview for DSM-5 (First, Williams, Karg, & Spitzer, 2015) confirmed schizophrenia-spectrum diagnosis, (c) owned a mobile phone that could send/receive text messages, (d) would permit study text messages be sent to their phone, (e) demonstrated at least a fourth grade reading level on the Graded Word List (La Pray & Ross, 1969), (f) were in a postacute illness phase (i.e., no past month inpatient hospitalizations or medication changes), and (g) had at least moderate motivation moderate motivation deficits on the Clinical Assessment Interview for Negative Symptoms (CAINS; Kring, Gur, Blanchard, Horan, & Reise, 2013) in a minimum of one domain: motivation for family, close friends and romantic relationships, work and school, and/or recreational activities.

Recruitment occurred over a span of eight months, and our recruitment targets were based on a number of factors. First, given that this was a feasibility and pilot study, we examined sample sizes in prior text-message intervention studies and pilot intervention work (Ben-Zeev, Kaiser, et al., 2014; Freeman et al., 2015; Granholm et al., 2012; Yanos, Roe, West, Smith, & Lysaker, 2012), whose enrolled samples ranged from 19 to 55. Based on our internal resources, we targeted 50 for the final sample. We also conducted a preliminary a priori power analysis that suggested this sample size would be sufficient to detect medium-large to large effect sizes, and other research has recommended 25 per condition for pilot randomized trials (Whitehead, Julious, Cooper, & Campbell, 2016). Finally, we allowed for some attrition. Specifically, we planned to enroll approximately 55 participants in the study and assumed a 10% attrition rate, resulting in approximately 50 participants completing the trial.

Procedure

After completing informed consent and baseline measures, each participant completed a goal-setting session with the study therapist, a doctoral student in clinical psychology, where they set recovery-oriented goals to complete over 8 weeks. Participants were then randomized to receive either (a) MEMS, or (b) no additional intervention (referred to hereafter as the control group). Randomization was conducted using a random number generator in blocks of 10; each block had an equal number of both conditions. Randomization codes were generated by an independent researcher and sealed in envelopes with consecutive numbers; these were opened in ascending order after participants completed baseline assessments and the goal-setting session. After randomization, study assessors only contacted (i.e., called) participants in the control condition during the 8 weeks to schedule the follow-up assessment. For participants in the MEMS group, we tried to limit additional contact over the 8 weeks outside of the text messages (detailed below) as much as possible to help ensure any identified effects were due to the text-messaging. Thus, following prior procedures (Ben-Zeev et al., 2014), the study therapist would only call participants in the MEMS group if they did not reply to study text messages for three consecutive days. Study assessors also called MEMS participants to schedule the follow-up assessment. Follow-up assessments were completed after 8 weeks for both groups. Participants completed all primary and secondary outcome measures at baseline and follow-up assessments with trained raters blinded to condition; the questionnaire assessing MEMS feasibility at the follow-up assessment was only provided to assessors and completed with participants after all other study measures and ratings had been completed. Participants were compensated $40 and could win an additional $2.00-$8.24 on a study task (see below) at each assessment. Text message costs were reimbursed ($30/month); to ensure this additional compensation was not differentially influencing outcomes, both groups received it. Study procedures were approved by the local institutional review board.

Goal-setting session. Prior to randomization, all participants set recovery goals to complete over eight weeks during a one-on-one goal-setting session with the study therapist that lasted approximately 45 min. Goals could be set in any domain, but participants were first asked if they wanted to make changes in the domains identified as reduced on the CAINS motivation items. The in-person goal-setting session incorporated techniques from collaborative goal technology (GCT; Clarke et al., 2006), a systematic, recovery goal-setting method focused on identifying the importance and personal meaning of a goal (Clarke et al., 2009). Using GCT and information gathered in the assessments, attempts were made to help participants integrate information to accurately identify and assess the value, effort, and probability of attaining an identified goal. Identified goals were translated into a specific, measurable, achievable, realistic, and timed (Bovend’Eerdt, Botell, & Wade, 2009; Schut & Stam, 1994) goal, and participants discussed and then rated the value/importance of the goal (rated from 1-10), effort required to complete the goal (rated from 1-10), and the participant’s confidence in completing the goal (rated 0-100%; a copy of this information was provided to participants). To further overcome effort-cost computation difficulties, for all participants, each overall goal was collaboratively broken down into smaller subgoals to complete each weekday over 8 weeks; these subgoals were then written on calendars, and copies were provided to each participant.

MEMS. Following prior studies (Ben-Zeev, Kaiser, et al., 2014; Granholm et al., 2012), the participants in the MEMS group received in-person training in text-messaging procedures before receiving study messages. Training lasted approximately 15 min and occurred with the participant and the study therapist who sent the text messages. First, the limits of text message confidentiality and ways to improve privacy (e.g., adding an access password) were reviewed. Next, participants were trained to send and receive text messages and modify relevant settings (e.g., text message notification volume, text font size) on their personal phone. Participants then engaged in a practice text-messaging session where they drafted and sent a message and opened and read a received message from the study therapist.

Participants in the MEMS group received three text message sets each weekday from the study therapist for eight weeks through TextIt’s (Nyaruka, 2016) web-based text-messaging service. Text messages were sent during three time blocks: (a) 8:30-10:30 a.m., (b) 11:30 a.m.-1:30 p.m., and (c) 5:30-7:30 p.m. Participants selected when they wanted to start receiving messages in each block and were informed that text messages sent outside the blocks may have a delayed reply. Following prior technology-based SMI research (Rotondi, Eack, Hanusa, Spring, & Haas, 2015), efforts were made to create text messages that required a low reading level and used concrete language.

The interactive text messages aimed to reinforce and cue goal completion and target effort-cost computations and future rewardvalue representation maintenance. Based on preset scripts that were individualized for each goal (see example daily text message exchange in Figure 1 in the online supplemental materials), daily messages occurred in the following order (relevant strategies are labeled with the behavior change technique taxonomy (v1); Michie et al., 2013): (a) reminder of the smaller subgoal they set to complete that day (i.e., goal-setting behavior/outcome [Michie et al., 2013]), inquiry about how much effort the goal would take to complete (on a scale of 1-10), and then positive encouragement to support effort expenditure; (b) encouragement that their subgoal is worth the effort, and reminder of why the goal is valuable to them (based on goal-setting session information), and inquiry about when they wanted to complete the goal that day (i.e., for action planning; Michie et al., 2013); and (c) assessment of subgoal completion and how much effort it took to complete the goal (on a scale of 1-10). If they did not complete the subgoal, participants were asked what might help them reach their subgoal and whether the subgoal could be broken down into smaller steps. If they did complete it, encouragement was provided to reinforce success and support adaptive effort-cost computations (i.e., if they overestimated the effort, then we reinforced that it was less effort then they anticipated). At the end of each week, feedback indicating progress toward their overall goal (i.e., feedback on outcomes; Michie et al., 2013) was provided. Thus, via text messages, effort-cost computations were primarily targeted through inquiry and assessments about subgoal effort, encouragement and positive reinforcement (i.e., social reward; Michie et al., 2013) supporting effort expenditure and adaptive effort-cost computations, as well as messages linking effort expenditure to rewards. Future reward-value representation maintenance was targeted through messages that provided reminders about and enhanced the connection between the value or rewards associated with subgoal completion.

Measures

Interviewer-rated motivation. The aforementioned CAINS four motivation items were used to assess motivation over the past week for the domains of family, close friends and romantic relationships, work and school, and recreational activities. The three-item Motivation Index (Choi, Choi, Felice Reddy, & Fiszdon, 2014; Nakagami, Xie, Hoe, & Brekke, 2008) from the Heinrichs-Carpenter Quality of Life Scale (QLS; Heinrichs, Hanlon, & Carpenter, 1984) was used to assess a person’s global degree of motivation to initiate and sustain activities (1 item), curiosity in daily life (1 item), and sense of purpose or having integrated, realistic life goals (1 item) over the preceding 4 weeks. Since the motivation item from the QLS-Motivation Index is also a valid stand-alone motivation measure (Fervaha, Foussias, Takeuchi, Agid, & Remington, 2015), we used this item in exploratory analyses. Both the CAINS and QLS scores have demonstrated good convergent validity and interrater reliability in schizophrenia-spectrum samples (Fervaha, Foussias, et al., 2015; Kring et al., 2013; Luther et al.,

Subjective motivation. The six motivation and effort items from the self-report Motivation and Pleasure Scale (MAP-SR; Llerena et al., 2013) were used to assess perceived motivation and effort over the past week for social, work, school, hobbies, and recreational activities. Items are rated on a variable 5-point Likert scale. Scores on the MAP-SR have demonstrated good convergent and discriminant validity and internal consistency (Llerena et al., 2013).

Effort-cost computations. Effort-cost computations were assessed by the EEfRT (Treadway et al., 2009), which contains trials where participants choose to complete an easy or hard task after viewing the associated monetary rewards for both options and probability of reward receipt. On this task, easy task rewards are always $1.00, while hard tasks rewards vary from $1.24-$4.12. The probability of reward receipt if the chosen task is completed varies (but is the same for each trial option), ranging from high (88%), medium (50%), to low (12%). The easy task asks participants to make 30 button presses in 7 s using their dominant-hand index finger, and the hard task requires 100 button presses in 21 s with their non-dominant-hand pinky finger. The task runs for 20-min, and participants are instructed that earnings from the task are based on two randomly selected tasks. Following prior methods (McCarthy et al., 2016), our main effort-cost computation outcome was the percentage of hard trials selected in the high reward (>$3.01) high probability (88%) trials. EEfRT scores have shown reliability and validity in schizophrenia samples (Green et al., 2015).

Future reward-value representations. Future reward-value representations were measured using a delay-discounting task (Kirby, Petry, & Bickel, 1999) where participants choose between a smaller immediate monetary reward or a larger delayed reward in 27 trials. For this task, small rewards range from $11-$80, while larger delayed rewards range from ($25-$85). Delays range from seven to 186 days. As studies have failed to find performance differences between hypothetical and real monetary rewards on delay discounting tasks (Bickel, Pitcock, Yi, & Angtuaco, 2009; Lagorio & Madden, 2005), participants were informed that they would not receive the rewards but should make their decisions as if the rewards were genuine. Following Myerson, Baumann, and Green (2014), greater ability to represent the value of a future reward was indexed by the percentage of larger delayed rewards selected.

MEMS usability and satisfaction. MEMS usability and satisfaction were assessed with 14 self-report items based on the Usability, Satisfaction, and Ease of Use Questionnaire (Lund, 2001), which was previously modified to assess the usability and satisfaction of a mobile intervention in a schizophrenia-spectrum sample (Ben-Zeev, Kaiser, et al., 2014). All items were rated on a 7-point Likert scale from 1 (Strongly Disagree) to 7 (Strongly Agree).

Functioning. Functioning was assessed with the total score of the interviewer-rated Strauss-Carpenter Level of Function scale (Hawk, Carpenter, & Strauss, 1975; Strauss & Carpenter, 1977). Nine items are rated on a 5-point variable scale and assess social contacts, work, symptoms, and general functioning over the past month. Scores on the scale have demonstrated interrater reliability and convergent validity in schizophrenia-spectrum samples (Strauss & Carpenter, 1977).

Quality of life. Quality of life was measured by the self-report overall quality of life item from the World Health Organization Quality of Life BREF Scale (WHOQOL-BREF; Skevington, Lotfy, O’Connell, & the WHOQOL Group, 2004). This item assesses quality of life over the past 2 weeks and is rated on a 5-point scale from 1 (very poor) to 5 (very good). The WHOQOL-BREF has demonstrated excellent convergent validity in a schizophrenia-spectrum sample (Mas-Exposito, Amador-Campos, Gomez-Benito, Lalucat-Jo, & the Research Group on Severe Mental Disorders, 2011).

Neurocognition. Neurocognition was measured using the updated Brief Neurocognitive Assessment (BNA; Fervaha, Hill, et al., 2015). The BNA assesses working memory with the letternumber sequencing test (Gold, Carpenter, Randolph, Goldberg, & Weinberger, 1997) and processing speed with the symbol coding subtest from the Brief Assessment of Cognition in Schizophrenia (Keefe et al., 2004). Following Fervaha, Hill, et al. (2015), we created an overall BNA standardized z score based on normative data. The BNA has demonstrated reliability and validity in schizophrenia samples (Fervaha, Hill, et al., 2015).

Additional negative symptoms. Anticipatory pleasure was measured with the three CAINS items assessing the frequency of expected pleasure for the upcoming week for the domains of social relationships, work and school, and recreational activities; the CAINS past week pleasure items (2 items) were also used to assess the frequency of pleasurable activities over the past week in the domains of social and recreational activities. Emotion expression and speech were assessed with the four expressive items (facial expression, vocal expression, expressive gestures, and quantity of speech).

Positive and mood symptoms. We used the factor-analytically derived positive (delusions, hallucinations, unusual thought content, somatic concern, suspiciousness/persecution, and grandiosity items) and emotional discomfort (i.e., mood; depression, anxiety, guilt feelings, and active social avoidance items) symptoms subscales (Bell, Lysaker, Beam-Goulet, Milstein, & Lindenmayer, 1994) on the widely used Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, 1987). Each interviewer-rated item is rated on a 7-point scale from 1 (Absent) to 7 (Severe). The PANSS scores have demonstrated satisfactory test-retest reliability and validity in schizophrenia-spectrum samples (Kay et al., 1987).

Analyses

For analyses, we used a full analysis set (United States Department of Health and Human Services, 1998), including data from all randomized participants, regardless of actual intervention use/ adherence. We first compared group demographics, motivation levels, and goal-setting ratings (e.g., effort to complete goals) using independent samples t tests and chi-square tests. Second, to assess MEMS feasibility and engagement, we examined text message response rates; descriptive statistics were used to assess responses to usability and satisfaction questions. Next, we used a series of analysis of covariances (ANCOVAs) to identify follow-up group differences on outcomes after covarying for the associated baseline outcome level and if necessary, any group demographic differences identified in the first step. We then compared the percentage of overall goals attained ([number of goals attained at follow up divided by number of goals sets at baseline] X 100) between groups using independent samples t tests; as a more stringent test, we also conducted an ANCOVA to compare the percentage of overall goals attained between groups after controlling for participants’ goal importance, effort, and confidence ratings. Finally, to identify whether MEMS engagement was related to outcome changes, correlations between text message response rate and outcome change scores (baseline minus follow up) were conducted. Effect sizes were categorized following Cohen (1992).

Results

Recruitment and Participant Characteristics

One hundred people were assessed for eligibility, and 56 were randomized (27 to MEMS, 29 to control). Three participants (5.4%) did not complete the study (see the CONSORT diagram in Figure 1). One participant in the MEMS group withdrew several weeks after starting the text messages because she obtained a job and thought she would not have time for the text messages; a second participant was administratively withdrawn after breaking her phone and becoming unreachable prior to beginning the text messages. One participant in the control group was unreachable at follow up.

At baseline, groups did not significantly differ on demographics (thus, demographics were not controlled for in additional analyses), number of goal-setting session goals set, or CAINS motivation (Table 1). However, participants in the MEMS group rated their overall goals as being more valuable/ important (p = .04) and requiring more effort (p = .02) to complete than the control group; groups did not differ in their confidence in achieving their goals. Information about goal domains and goal examples is in Table 1 in the online supplemental materials. In the full sample, motivation deficits were moderate (M = 7.6, SD = 2.3), and participants set an average of 3.6 (SD = 1.7) overall goals for the 8 weeks; most had unlimited text messages in their service plan (96.4%). Study noncompleters (n = 3) and completers (n = 53) did not significantly differ on demographics or CAINS motivation.

MEMS Feasibility, Engagement, Usability, and Satisfaction

Feasibility and engagement. Across the 8 weeks, participants received an average of 207.5 (SD = 62.4) study text messages and sent an average of 185.8 (SD = 92.6) study text messages. Average participant response rate was 86.1% (SD = 16.7%); one participant responded to 18.5%, three responded to 63.1%-73.3%, nine responded to 80-89.4%, and 12 responded to over 93% of the text messages.

Usability and satisfaction. Regarding usability, 96% (n = 24) of participants who received MEMS reported they learned MEMS quickly and it was easy to use (Table 2). Sixteen percent (n = 4) reported difficulties understanding the text messages and typing their responses, and 12% (n = 3) reported difficulties operating their phone. For satisfaction, all participants reported they were satisfied with the text messages, and 92% (n = 23) reported the text messages were useful and helped them to become more motivated. Ninety-two percent (n = 23) reported the text messages helped them to reach their goals and get more things done. Several participants also made unprompted text message comments about how MEMS helped them (See Table 2 in the online supplemental materials).

Preliminary Effectiveness

Primary outcomes. Consistent with hypotheses, significant medium-sized group effects were found for CAINS motivation (p = .03; Table 3); after controlling for CAINS motivation at baseline, participants who received MEMS demonstrated greater motivation at follow up than participants in the control group. No significant group effects were found for the QLS-Motivation Index (p = .14), but in exploratory analyses, the participants in the MEMS group had greater follow-up scores on the motivation item of the index than the participants in the control group after adjusting for baseline scores (p = .04); effect size was medium. As hypothesized, participants in the MEMS group reached a significantly greater percentage of overall goals than participants in the control group (p < .001), with a large effect size. This difference in overall goal attainment (as well as the magnitude of the effect size) remained after accounting for participants’ goal value/importance, effort, and confidence ratings (p = .001). Contrary to hypotheses, no significant group effects were found for subjective motivation (p = .61), future reward-value representations (p = .33), or


effort-cost computations (p = .70); however, several participants demonstrated baseline ceiling effects or fixed responses on the EEfRT (n = 12) as well as fixed or inconsistent responses (Kirby et al., 1999; R. Yu, 2012) on the delay discounting task (n = 7; results were statistically the same when these participants were excluded).

Secondary outcomes. After controlling for baseline levels, anticipatory pleasure at follow up was significantly higher for participants in the MEMS group compared to those in the control group (p = .02), with a medium effect size (see Table 3). There was also a trend toward higher past week pleasure (p = .096) at follow-up for participants in the MEMS group relative to those in the control condition. There were no significant group differences for expressive negative symptoms, positive symptoms, mood symptoms, neurocognition, quality of life, or functioning at follow up (all ps > .61).

MEMS engagement and outcome change. Among participants who received MEMS, a higher text message response rate was significantly associated with greater improvement in effortcost computations (p = .001) and anticipatory pleasure (p = .03). No other correlations were significant (Table 4).

Discussion

In a small, pilot randomized controlled trial, we tested the feasibility and preliminary effectiveness of MEMS, a motivation treatment delivered via mobile technology and text-messaging that was designed to target reward-processing mechanisms purported to underly reduced motivation. Our results show that MEMS is feasible and may lead to significantly greater improvements in interviewer-rated motivation, anticipatory pleasure, and recovery-oriented goal attainment than a goal-setting session alone. To our knowledge, this study is the first randomized controlled trial demonstrating the feasibility of solely using participants’ personal cellphones (rather than study provided cellphones) to deliver an interactive mobile intervention in those with schizophrenia-spectrum disorders, supporting the ecological validity, scalability, and real-world implementation of text-messaging interventions for this population.

Building on growing literature suggesting that text-messaging interventions are feasible and acceptable for most people with schizophrenia (Depp et al., 2010; Naslund et al., 2015), we found that MEMS was highly engaging for most participants. Across the 8-week intervention, the retention rate among participants who received

Table 1

Baseline Participant Demographics by Group

Demographic

MEMS

(n = 27), n (%)

Goal setting alone

(n = 29), n (%o)

Test of significance

Diagnosis

X2(1) = .25

Schizophrenia

12 (44.4)

11 (37.9)

Schizoaffective disorder

15 (55.6)

18 (62.1)

Gender (n, % female)

15 (55.6)

12(41.4)

X2(1) = 1.13

Race

X2(2) = .78

African American

18 (66.7)

21 (72.4)

White

8 (29.6)

6 (20.7)

Other or multiple races

1 (3.7)

2 (6.9)

Unlimited text message plan

26 (96.3)

28 (96.6)

X2(1) = .003

M

SD

Age

46.0 (10.0)

46.3 (7.7)

t(54) = - 0.12

Education

12.0 (2.7)

11.7 (2.0)

t(54) = .35

Chlorpromazine equivalent dosesa

618.3 (544.6)

416.1 (376.5)

t(54) = 1.63

Length of illness

24.0 (12.1)b

23.4 (10.5)

t(52) = .21

CAINS—Motivation

7.7 (2.6)

7.5 (1.9)

t(54) = .30

Number of goals set

3.7 (1.8)

3.5 (1.6)

t(54) = .48

Value/importance of goal(s)c

9.3 (1.0)

8.5 (1.9)

t(54) = 2.10*

Effort to complete goal(s)c

8.8 (1.4)

7.7 (2.3)

t(54) = 2.36*

Confidence in completing goal(s)c

84.1 (20.0)

81.9 (16.5)

t(54) = .45

Note. MEMS = Mobile Enhancement of Motivation in Schizophrenia; CAINS = Clinical Assessment Interview for Negative Symptoms.

a Based on prior studies (Herz et al., 1997; Woods, 2003; Woods, 2011). b Data missing for two participants. c These were based on ratings completed during the goal-setting session.

* p < .05.

MEMS was 92.6%, and the overall mean text message response rate was 86.1%. Although these rates are similar to prior text-messaging intervention studies (Ben-Zeev, Kaiser, et al., 2014; Granholm et al., 2012; Montes et al., 2012; Pijnenborg et al., 2010), our findings are particularly noteworthy since all participants demonstrated at least moderate baseline motivation deficits and were using personal cellphones. In addition, all participants reported satisfaction with the text messages, and almost all reported that the text messages were useful and increased their motivation and that MEMS was easy to use. Several participants also provided unprompted feedback that the text messages were encouraging, motivating, and helpful.

This study builds on feasibility and acceptability studies by using a randomized design to more rigorously test the preliminary effectiveness of MEMS. Results demonstrated that MEMS led to greater

Table 2

Usability and Satisfaction for MEMS Participants

Item

Strongly disagree

Disagree

Disagree somewhat

Neutral

Agree somewhat

Agree

Strongly agree

Usability items

I learned to use the mobile intervention quickly.

0

1 (4%)

0

0

4 (16%)

8 (32%)

12 (48%)

The mobile intervention was easy to use.

1 (4%)

0

0

0

1 (4%)

12 (48%)

11 (44%)

The mobile intervention did everything I would expect it to.

0

0

0

3 (12%)

4 (16%)

8 (32%)

10 (40%)

I had difficulties typing my responses.

14 (56%)

3 (12%)

1 (4%)

3 (12%)

2 (8%)

0

2 (8%)

I had difficulties operating my phone.

15 (60%)

4 (16%)

1 (4%)

2 (8%)

1 (4%)

2 (8%)

0

I had difficulties understanding the text messages.

16 (64%)

2 (8%)

2 (8%)

1 (4%)

0

2 (8%)

2 (8%)

The text messages interfered with my daily activities. Satisfaction items

17 (68%)

3 (12%)

1 (4%)

2 (8%)

2 (8%)

0

0

The text messages I received were useful.

0

0

0

1 (4%)

1 (4%)

7 (28%)

16 (64%)

I was satisfied with the text messages I received.a

0

0

0

0

2 (8%)

4 (16%)

18 (72%)

I would be interested in participating in similar studies in the future. I would recommend to others that they should participate in a

0

0

0

0

0

6 (24%)

19 (76%)

similar study.

0

0

0

0

2 (8%)

7 (28%)

16 (64%)

The text messages helped me to reach my goal(s).

0

0

0

2 (8%)

2 (8%)

6 (24%)

15 (60%)

The text messages helped me to get more things done.

0

0

1 (4%)

1 (4%)

2 (8%)

5 (20%)

16 (64%)

The text messages helped me become more motivated.a

0

0

0

1 (4%)

2 (8%)

7 (28%)

14 (56%)


Note. n = 25. MEMS = Mobile Enhancement of Motivation in Schizophrenia. a n = 1 (4%) missing data for this item.


Table 3

Measure Descriptive Statistics and Group Effects for Primary and Secondary Outcomes

MEMS,                Goal setting alone,

M (SD)                 M (SD)

Measure

BL

(n = 27)

8 week (n = 25)

BL

(n = 29)

8 week (n = 28)

F3

P

db [95% CI]

Primary outcomes

CAINS: Motivationc

7.7 (2.6)

6.2 (2.5)

7.5 (1.9)

7.4 (2.7)

4.73

.03

-0.58 [-1.14, -.03]

QLS- Motivation Index

8.0 (2.5)

9.6 (3.8)

7.4 (2.8)

8.0 (3.5)

2.23

.14

0.41 [-.14, .95]

QLS: Motivation item

2.5 (1.3)

3.6 (1.4)

2.3 (1.1)

2.9 (1.3)

4.59

.04

0.58 [.03, 1.13]

MAP-SR—Motivation

11.2(5.4)

11.6(5.6)

7.9 (5.0)

10.3 (6.5)

0.26

.61

-0.14 [-.68, .40]

Overall goals attainedd

77.6 (26.7)

46.7 (31.6)

3.82d

<.001

1.05e [.46, 1.61]

Value representation maintenance: %

delayed rewards

35.7 (21.2)

32.7 (19.2)

28.6 (23.0)

30.3 (25.4)

0.96

.33

-0.27 [-.81, .27]

Effort-cost computations: % hard chosen

in 88%, high reward trials

45.9 (32.4)

42.0 (35.5)

36.6 (29.2)

38.9 (36.4)

0.15

.70

-0.11 [-.65, .43]

Secondary outcomes

CAINS: Anticipatory pleasure

6.8 (3.3)

5.3 (2.5)

7.8 (2.8)

7.2 (2.5)

5.93

.02

-0.66 [-1.22, -.11]

CAINS: Past week pleasure

3.6 (2.3)

2.6 (1.9)

4.0 (2.0)

3.4 (1.4)

2.87

.096

-0.46 [-1.01, .08]

CAINS: Expressive symptoms

5.1 (3.3)

4.4 (3.9)

6.0 (4.0)

5.2 (3.4)

0.26

.62

-0.14 [-.68, .40]

PANSS: Positive symptoms

3.2 (.9)

2.7 (1.0)

2.9 (.8)

2.5 (.8)

0.02

.89

-0.04 [-.58, .50]

PANSS: Mood symptoms

3.3 (1.1)

3.0 (1.2)

3.0 (1.1)

2.9 (1.1)

0.01

.94

-0.02 [-.56, .52]

BNA: Neurocognition

-1.7 (1.2)

-1.7 (1.0)

-1.8 (1.1)

-1.6 (1.0)

0.12

.73

-0.10 [-.64, .44]

WHOQOL: Overall QOL

3.4 (1.0)

3.6 (1.0)

3.2 (1.2)

3.6 (1.0)

0.08

.78

-0.07 [-.61, .47]

Strauss-Carpenter: Functioning

16.9 (5.5)

19.4 (4.5)

17.0 (4.8)

19.0 (4.8)

0.11

.74

0.09 [-.45, .63]

Note. Descriptive statistics are simple statistics without co-varying for baseline level of variable. MEMS = Mobile Enhancement of Motivation in Schizophrenia; BL = baseline; CAINS = Clinical Assessment Interview for Negative Symptoms; MAP-SR = Motivation and Pleasure Self-Report; PANsS = Positive and Negative Syndrome Scale; QLS = Quality of Life Scale; QOL = quality of life; WHOQOL = World Health Organization Quality of Life.

a Results based on those who completed both assessment points. b Unless otherwise noted, effect sizes were calculated with adjusted follow-up means and pooled standard deviations. c Higher CAINS and PANSS scores = more symptoms (e.g., greater motivation deficits). Higher BNA, WHOQOL, and Strauss-Carpenter = better neurocognition, QOL, or functioning, respectively. d t value and associated significance test and effect size (based on follow-up means and pooled standard deviations) are reported. e Effect size when controlling for participants’ goal importance, effort, and confidence ratings was .98 [.40, 1.54].


improvements in interviewer-rated motivation, anticipatory pleasure, and recovery-oriented goal attainment compared to a goal-setting session, with medium to large effect sizes. Thus, MEMS appeared to improve more objective behavioral components of motivation such as the initiation and maintenance of behaviors that support meaningful goal attainment and the completion of daily activities as well as some internal aspects related to motivation such as one’s interest in activities and expectations about the pleasure derived from prospective activities. Although larger trials are needed to confirm these effects, these findings are promising given that there are few relatively brief treatments (i.e., <18 months) that have demonstrated efficacy for improving motivation and anticipatory anhedonia in prolonged schizophrenia. Indeed, this aligns with work suggesting that compared to usual in-person care, mobile interventions may offer a more sustainable, scalable, and potentially cost-effective treatment approach (De La Torre-Diez, Lopez-Coronado, Vaca, Aguado, & de Castro, 2015; Depp et al., 2010).

In addition, the observed effects of MEMS on recovery-oriented goal attainment did not seem to be due to group differences in goal ratings of importance and effort. Indeed, at baseline, participants in the MEMS group rated their goals as being more valuable/impor-tant but also requiring more effort than those in control group, while we found no group differences in participants’ reported confidence in achieving their goals. Prior research has shown that perceptions of goal importance and difficulty as well as confidence in completing a goal can impact goal attainment (Locke & Latham, 2002; Clarke et al., 2006). Thus, we also controlled for baseline participant goal ratings of value/importance, effort, and confidence in completing their goal(s) when comparing group overall goal attainment, and the group differences in overall attainment and the large effect size remained. Although it is possible that setting more important and valuable goals could have helped the participants in the MEMS group to achieve more overall goals than the control group, our results suggest that the group differences in goal attainment remained even after accounting for participants’ ratings of value/importance as well as the effort and confidence in achieving their goal(s).

However, in contrast to our hypotheses, we did not find that MEMS was more effective at improving effort-cost computations, future reward-value representations, or self-reported motivation than a goal-setting session. This lack of findings for the performance-based tasks was particularly surprising because these tasks were putatively assessing the targeted mechanisms through which we expected motivation to improve; we speculate this may be due to power limitations or the near ceiling level or fixed responses at baseline for several participants, with the latter issues suggesting that these performance-based tasks have limited utility as outcome measures in clinical trials. Alternatively, these tasks may have not effectively represented the constructs we were targeting in MEMS (e.g., were too different, distal) or the tasks may have been “too easy,” particularly in comparison to real-world goals that generally require greater effort than button presses.

Table 4

Correlations Between MEMS Engagement and Outcome Change

Measure

r

CAINS: Motivation

0.20

QLS-Motivation Index

-0.07

QLS: Motivation item

-0.28

MAP-SR: Motivation

-0.31

Value representation maintenance: % delayed rewards

-0.26

Effort-cost computations: % hard chosen in 88%, high

reward trials

-0.61“

CAINS: Anticipatory pleasure

0.43*

CAINS: Past week pleasure

0.21

CAINS: Expressive symptoms

0.21

PANSS: Positive symptoms

-0.09

PANSS: Mood symptoms

-0.06

BNA: Neurocognition

-0.17

WHOQOL: Overall QOL

-0.24

Strauss-Carpenter: Functioning

-0.09

Note. n = 25. For Clinical Assessment Interview for Negative Symptoms (CAINS) and Positive and Negative Syndrome Scale (PANSS), positive correlation = higher response rate associated with greater reduction in symptoms. For other measures, negative correlation = higher response rate is associated with greater improvement in measure. MEMS = Mobile Enhancement of Motivation in Schizophrenia; BNA = Brief Neurocognitive Assessment; MAP-SR = Motivation and Pleasure Self-Report; WHO-QOL = World Health Organization Quality of Life; QLS = Quality of Life Scale; QOL = quality of life.

* p < .05. ** p < .01.

Further, this is the first study to our knowledge to use the EEfRT in a clinical trial. Although we chose our primary EEfRT outcome score based on what we believed best aligned with the mechanism we were targeting, given that prior studies have used a range of EEfRT administration modifications and scoring methods (Luther, Firmin, Lysaker, Minor, & Salyers, 2018), additional work could examine whether alternative EEfRT administration and scoring methods may have greater utility and sensitivity in clinical trials. However, importantly, contrasting prior work guiding our decision to select the EEfRT (Barch et al., 2014; Fervaha et al., 2013), recent work has found limited overlap between the EEfRT and the CAINS and QLS-Motivation Index (Luther, Fischer, Firmin, & Salyers, 2019; Luther et al., 2018), suggesting they may measure disparate constructs. To better assess mechanisms of MEMS improvement and to more precisely identify whether effort-cost computations and future reward-value representation maintenance are effective motivational enhancement treatment targets, future work could use more recent performance-based tasks, such as effort discounting tasks (Hartmann et al., 2015), which have shown greater concordance with motivation/negative symptom measures (Luther et al., 2019).

For secondary outcomes, there were significantly greater improvements in anticipatory pleasure for those who received MEMS compared to those in the control group (medium effect size). Past week pleasure also trended toward greater improvement for those in the MEMS group relative to participants in the control group. It may be that as participants worked more regularly toward their goals or had more success attaining subgoals, they had greater anticipated and experienced enjoyment for goal-related activities. The text message reminders about why the subgoals were worth the effort and valuable could also have helped participants to more readily represent future rewards such as pleasure as well as strengthen the mental link between subgoal completion and future rewards, leading to greater anticipated pleasure (Heerey & Gold, 2007). These results suggest that MEMS may reduce the consummatory-anticipatory pleasure gap found in those with schizophrenia (Gard, Kring, Gard, Horan, & Green, 2007). However, there were no significant group differences in the secondary outcomes of positive, mood, and expressive symptoms or neurocognition, quality of life, and functioning. Longer-term studies may help to determine whether the identified effects of MEMS translate into improvements in these more distal symptoms or broader outcomes.

It is possible that the observed improvements related to MEMS in interviewer-rated motivation and recovery-oriented goal attainment as well as anticipatory pleasure were due to alternative mechanisms of change. Although speculative, one possibility is that the social interaction and accountability provided through the text messages may have led to the observed improvements. Indeed, in line with Self-Determination Theory (Ryan & Deci, 2000), which suggests that a sense of connection and belonging is an important component for fostering well-being and motivation, it is possible that interacting daily with someone and receiving more social support via text messages helped to improve interviewerrated motivation and goal-attainment. It is also possible that the daily text message reminders about participants’ subgoals and the anticipated inquiry about subgoal completion could have by themselves helped to directly improve these outcomes. Alternatively, other factors that have been linked to reduced motivation and pleasure such as defeatist beliefs (Grant & Beck, 2009) or decreased competence (Ryan & Deci, 2000) may have been improved by MEMS. It may be that the text messages encouraging participants to engage in activities to support their goals combined with reminders about why the activity is worth the effort and valuable could have reduced defeatist attitudes about goal-related activities (e.g., “It is too hard or will take too much effort, so why try”; “I can’t do this well, so why try at all.”). Similarly, reinforcing successful goal completion and providing feedback about discrepancies between participants’ anticipated effort and actual effort for their daily subgoal, especially when subgoals were easier than expected, could also have helped to improve participants’ sense of competence and expectancies of future successful goal completion and associated rewards such as pleasure. Together, MEMS could have resulted in improvements in defeatist thinking and/or competence, which in turn led to improvements in interviewer-rated motivation, recovery-goal attainment, and anticipatory pleasure. Future work is needed to identify whether these alternative mechanisms of change account for the observed effects of MEMS or play a role in other mobile interventions, particularly those targeting motivation reductions or other negative symptoms.

Exploratory analyses revealed that higher MEMS engagement was associated with greater improvements in effort-cost computations and anticipatory pleasure. For effort-cost computations, this may suggest that only those with higher MEMS engagement saw improvements on this domain. Alternatively, the goal-setting session and breaking down overall goals into daily subgoals could have helped to improve effort-cost computations in both groups, obscuring the additional benefits of MEMS on effort-cost computations when conducting group comparisons. Relatedly, MEMS engagement was not significantly associated with interviewerrated motivation improvements; however, the correlation magnitudes between engagement and the CAINS motivation items, the QLS-motivation item, and self-reported motivation on the MAP-SR were small to medium, further supporting the need for a future trial with a larger sample to provide better estimates of these effects.

Several limitations should be considered. First, we did not examine whether MEMS group improvements were maintained over time. Second, although consistent with or larger than prior text-messaging studies (Ben-Zeev, Kaiser, et al., 2014; Pijnenborg et al., 2010), our sample was relatively small and may have been underpowered to detect some effects; thus, additional work with larger samples is needed to confirm the observed effects. The use of personalized text messages may also pose a challenge for widespread dissemination, given the need for clinical personnel. However, more automated approaches are now available, and future studies could compare the efficacy of a completely automated approach to our more personalized clinician-based approach. Further, although we found that the large majority of participants with at least moderate motivation reductions had a personal cell phone, it may be that those with more significant motivation reductions may be less likely to own a personal cell phone, ultimately impacting the feasibility and scalability of using personal cell phones to deliver mobile interventions. Relatedly, future work is needed to identify who may be most likely to engage and benefit in mobile interventions and identify what factors might impact engagement when mobile interventions are delivered on personal cell phones. Also, we reimbursed text message costs; however, almost all participants had unlimited text message service plans, suggesting reimbursement may not be needed in future work. Finally, although study procedures were in place to have outcome assessors be blind to study conditions, we did not assess whether the blind was broken during follow-up assessments.

Our findings highlight the feasibility of using personal cellphones to deliver text-messaging interventions to support those with schizophrenia-spectrum disorders in real-time, real-world settings. Although more work with higher powered samples is needed to further examine the precise effects of MEMS, our results indicate that MEMS may be an efficacious mobile treatment to improve one of the most debilitating symptoms of schizophrenia— motivation deficits—as well as help participants attain meaningful life-goals supporting their recovery.

References

Ahn, W.-Y., Rass, O., Fridberg, D. J., Bishara, A. J., Forsyth, J. K., Breier, A., . . . O'Donnell, B. F. (2011). Temporal discounting of rewards in patients with bipolar disorder and schizophrenia. Journal of Abnormal Psychology, 120, 911-921. http://dx.doi.org/10.1037/a0023333

Barch, D. M., Treadway, M. T., & Schoen, N. (2014). Effort, anhedonia, and function in schizophrenia: Reduced effort allocation predicts amo-tivation and functional impairment. Journal of Abnormal Psychology, 123, 387-397. http://dx.doi.org/10.1037/a0036299

Bell, M. D., Lysaker, P. H., Beam-Goulet, J. L., Milstein, R. M., & Lindenmayer, J.-P. (1994). Five-component model of schizophrenia: Assessing the factorial invariance of the Positive and Negative Syndrome Scale. Psychiatry Research, 52, 295-303. http://dx.doi.org/10 .1016/0165-1781(94)90075-2

Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014). Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophrenia Bulletin, 40, 1244-1253. http://dx.doi.org/10.1093/schbul/sbu033

Ben-Zeev, D., Davis, K. E., Kaiser, S., Krzsos, I., & Drake, R. E. (2013). Mobile technologies among people with serious mental illness: Opportunities for future services. Administration and Policy in Mental Health and Mental Health Services Research, 40, 340-343. http://dx.doi.org/ 10.1007/s10488-012-0424-x

Ben-Zeev, D., Kaiser, S. M., & Krzos, I. (2014). Remote “hovering” with individuals with psychotic disorders and substance use: Feasibility, engagement, and therapeutic alliance with a text-messaging mobile interventionist. Journal of Dual Diagnosis, 10, 197-203. http://dx.doi .org/10.1080/15504263.2014.962336

Bickel, W. K., Pitcock, J. A., Yi, R., & Angtuaco, E. J. (2009). Congruence of BOLD response across intertemporal choice conditions: Fictive and real money gains and losses. The Journal of Neuroscience, 29, 88398846. http://dx.doi.org/10.1523/JNEUROSCI.5319-08.2009

Bovend'Eerdt, T. J., Botell, R. E., & Wade, D. T. (2009). Writing SMART rehabilitation goals and achieving goal attainment scaling: A practical guide. Clinical Rehabilitation, 23, 352-361. http://dx.doi.org/10.1177/ 0269215508101741

Choi, J., Choi, K. H., Felice Reddy, L., & Fiszdon, J. M. (2014). Measuring motivation in schizophrenia: Is a general state of motivation necessary for task-specific motivation? Schizophrenia Research, 153, 209 -213. http://dx.doi.org/10.1016/j.schres.2014.01.027

Clarke, S. P., Oades, L. G., Crowe, T. P., Caputi, P., & Deane, F. P. (2009). The role of symptom distress and goal attainment in promoting aspects of psychological recovery for consumers with enduring mental illness. Journal of Mental Health (Abingdon, England), 18, 389-397. http://dx .doi.org/10.3109/09638230902968290

Clarke, S. P., Oades, L. G., Crowe, T. P., & Deane, F. P. (2006). Collaborative goal technology: Theory and practice. Psychiatric Rehabilitation Journal, 30, 129-136. http://dx.doi.org/10.2975/30.2006.129.136

Cloutier, M., Sanon Aigbogun, M., Guerin, A., Nitulescu, R., Ramanaku-mar, A. V., Kamat, S. A., . . . Wu, E. (2016). The Economic Burden of Schizophrenia in the United States in 2013. The Journal of Clinical Psychiatry, 77, 764-771. http://dx.doi.org/10.4088/JCP.15m10278

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. http://dx.doi.org/10.1037/0033-2909.112.E155

de la Torre-Diez, I., Lopez-Coronado, M., Vaca, C., Aguado, J. S., & de Castro, C. (2015). Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: A systematic review. Telemedicine and e-Health, 21, 81-85. http://dx.doi.org/10 .1089/tmj.2014.0053

Depp, C. A., Mausbach, B., Granholm, E., Cardenas, V., Ben-Zeev, D., Patterson, T. L., . . . Jeste, D. V. (2010). Mobile interventions for severe mental illness: Design and preliminary data from three approaches. Journal of Nervous and Mental Disease, 198, 715-721. http://dx.doi .org/10.1097/NMD.0b013e3181f49ea3

Fervaha, G., Agid, O., Takeuchi, H., Foussias, G., & Remington, G. (2013). Clinical determinants of life satisfaction in chronic schizophrenia: Data from the CATIE study. Schizophrenia Research, 151, 203208. http://dx.doi.org/10.1016/j.schres.2013.10.021

Fervaha, G., Foussias, G., Agid, O., & Remington, G. (2014). Motivational and neurocognitive deficits are central to the prediction of longitudinal functional outcome in schizophrenia. Acta Psychiatrica Scandinavica, 130, 290-299. http://dx.doi.org/10.1111/acps.12289

Fervaha, G., Foussias, G., Takeuchi, H., Agid, O., & Remington, G. (2015). Measuring motivation in people with schizophrenia. Schizophrenia Research, 169, 423-426. http://dx.doi.org/10.1016/j.schres.2015.09 .012

Fervaha, G., Graff-Guerrero, A., Zakzanis, K. K., Foussias, G., Agid, O., & Remington, G. (2013). Incentive motivation deficits in schizophrenia reflect effort computation impairments during cost-benefit decisionmaking. Journal of Psychiatric Research, 47, 1590-1596. http://dx.doi .org/10.1016/j.jpsychires.2013.08.003

Fervaha, G., Hill, C., Agid, O., Takeuchi, H., Foussias, G., Siddiqui, I.,.. . Remington, G. (2015). Examination of the validity of the Brief Neuro-cognitive Assessment (BNA) for schizophrenia. Schizophrenia Research, 166, 304-309. http://dx.doi.org/10.1016/j.schres.2015.05.015

First, M. B., Williams, J., Karg, R. S., & Spitzer, R. L. (2015). User’sguide to Structured Clinical Interview for DSM-5 Disorders (SCID-5-CV) clinical version. Arlington, VA: American Psychiatric Association.

Freeman, D., Waite, F., Startup, H., Myers, E., Lister, R., McInerney, J., ... Yu, L. M. (2015). Efficacy of cognitive behavioural therapy for sleep improvement in patients with persistent delusions and hallucinations (BEST): A prospective, assessor-blind, randomised controlled pilot trial. The Lancet Psychiatry, 2, 975-983. http://dx.doi.org/10.1016/S2215-0366(15)00314-4

Fusar-Poli, P., Papanastasiou, E., Stahl, D., Rocchetti, M., Carpenter, W., Shergill, S., & McGuire, P. (2015). Treatments of Negative Symptoms in Schizophrenia: Meta-Analysis of 168 Randomized Placebo-Controlled Trials. Schizophrenia Bulletin, 41, 892-899. http://dx.doi .org/10.1093/schbul/sbu170

Gard, D. E., Cooper, S., Fisher, M., Genevsky, A., Mikels, J. A., & Vinogradov, S. (2011). Evidence for an emotion maintenance deficit in schizophrenia. Psychiatry Research, 187, 24-29. http://dx.doi.org/10 .1016/j.psychres.2010.12.018

Gard, D. E., Kring, A. M., Gard, M. G., Horan, W. P., & Green, M. F. (2007). Anhedonia in schizophrenia: Distinctions between anticipatory and consummatory pleasure. Schizophrenia Research, 93, 253-260. http://dx.doi.org/10.1016/j.schres.2007.03.008

Gold, J. M., Carpenter, C., Randolph, C., Goldberg, T. E., & Weinberger, D. R. (1997). Auditory working memory and Wisconsin Card Sorting Test performance in schizophrenia. Archives of General Psychiatry, 54, 159-165. http://dx.doi.org/10.1001/archpsyc.1997.01830140071013

Gold, J. M., Waltz, J. A., Prentice, K. J., Morris, S. E., & Heerey, E. A. (2008). Reward processing in schizophrenia: A deficit in the representation of value. Schizophrenia Bulletin, 34, 835-847. http://dx.doi.org/ 10.1093/schbul/sbn068

Granholm, E., Ben-Zeev, D., Link, P. C., Bradshaw, K. R., & Holden, J. L. (2012). Mobile Assessment and Treatment for Schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophrenia Bulletin, 38, 414-425. http://dx.doi.org/10.1093/schbul/sbr155

Grant, P. M., & Beck, A. T. (2009). Defeatist beliefs as a mediator of cognitive impairment, negative symptoms, and functioning in schizophrenia. Schizophrenia Bulletin, 35, 798-806. http://dx.doi.org/10 .1093/schbul/sbn008

Green, M. F., Horan, W. P., Barch, D. M., & Gold, J. M. (2015). Effortbased decision making: A novel approach for assessing motivation in schizophrenia. Schizophrenia Bulletin, 41, 1035-1044. http://dx.doi.org/ 10.1093/schbul/sbv071

Harding, B., Torres-Harding, S., Bond, G. R., Salyers, M. P., Rollins, A. L., & Hardin, T. (2008). Factors associated with early attrition from psychosocial rehabilitation programs. Community Mental Health Journal, 44, 283-288. http://dx.doi.org/10.1007/s10597-008-9128-9

Hartmann, M. N., Hager, O. M., Reimann, A. V., Chumbley, J. R., Kirschner, M., Seifritz, E., . . . Kaiser, S. (2015). Apathy but not diminished expression in schizophrenia is associated with discounting of monetary rewards by physical effort. Schizophrenia Bulletin, 41, 503512. http://dx.doi.org/10.1093/schbul/sbu102

Hawk, A. B., Carpenter, W. T., Jr., & Strauss, J. S. (1975). Diagnostic criteria and five-year outcome in schizophrenia. A report from the International Pilot Study of Schizophrenia. Archives of General Psychiatry, 32, 343-347. http://dx.doi.org/10.1001/archpsyc.1975.017 60210077005

Heerey, E. A., & Gold, J. M. (2007). Patients with schizophrenia demonstrate dissociation between affective experience and motivated behavior. Journal of Abnormal Psychology, 116, 268 -278. http://dx.doi.org/10 .1037/0021-843X.116.2.268

Heerey, E. A., Matveeva, T. M., & Gold, J. M. (2011). Imagining the future: Degraded representations of future rewards and events in schizophrenia. Journal of Abnormal Psychology, 120, 483-489. http://dx.doi .org/10.1037/a0021810

Heerey, E. A., Robinson, B. M., McMahon, R. P., & Gold, J. M. (2007). Delay discounting in schizophrenia. Cognitive Neuropsychiatry, 12, 213-221. http://dx.doi.org/10.1080/13546800601005900

Heinrichs, D. W., Hanlon, T. E., & Carpenter, W. T., Jr. (1984). The Quality of Life Scale: An instrument for rating the schizophrenic deficit syndrome. Schizophrenia Bulletin, 10, 388-398. http://dx.doi.org/10 .1093/schbul/10.3.388

Herz, M. I., Liberman, R. P., McGlashan, T. H., Wyatt, R. J., Marder, S. R., Wang, P., . .. Zonana, H. V. (1997). Practice guideline for the treatment of patients with schizophrenia. The American Journal of Psychiatry, 154, 1-63. http://dx.doi.org/10.1176/ajp.154A1

Horan, W. P., Reddy, L. F., Barch, D. M., Buchanan, R. W., Dunayevich, E., Gold, J. M., . . . Green, M. F. (2015). Effort-based decision-making paradigms for clinical trials in schizophrenia: Part 2—External validity and correlates. Schizophrenia Bulletin, 41, 1055-1065. http://dx.doi.org/ 10.1093/schbul/sbv090

Kannisto, K. A., Adams, C. E., Koivunen, M., Katajisto, J., & Valimaki, M. (2015). Feedback on SMS reminders to encourage adherence among patients taking antipsychotic medication: A cross-sectional survey nested within a randomised trial. British Medical Journal Open, 5, e008574. http://dx.doi.org/10.1136/bmjopen-2015-008574

Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13, 261-276. http://dx.doi.org/10.1093/schbul/13.2.261

Keefe, R. S., Goldberg, T. E., Harvey, P. D., Gold, J. M., Poe, M. P., & Coughenour, L. (2004). The Brief Assessment of Cognition in Schizophrenia: Reliability, sensitivity, and comparison with a standard neuro-cognitive battery. Schizophrenia Research, 68(2-3), 283-297. http://dx .doi.org/10.1016/j.schres.2003.09.011

Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal ofExperimentalPsychology: General, 128, 78-87. http://dx.doi .org/10.1037/0096-3445.128.1.78

Kring, A. M., Gur, R. E., Blanchard, J. J., Horan, W. P., & Reise, S. P. (2013). The Clinical Assessment Interview for Negative Symptoms (CAINS): Final development and validation. The American Journal of Psychiatry, 170, 165-172. http://dx.doi.org/10.1176/appi.ajp.2012 .12010109

Kurtz, M. M., Rose, J., & Wexler, B. E. (2011). Predictors of participation in community outpatient psychosocial rehabilitation in schizophrenia. Community Mental Health Journal, 47, 622-627. http://dx.doi.org/10 .1007/s10597-010-9343-z

Lagorio, C. H., & Madden, G. J. (2005). Delay discounting of real and hypothetical rewards III: Steady-state assessments, forced-choice trials, and all real rewards. Behavioural Processes, 69, 173-187. http://dx.doi .org/10.1016/j.beproc.2005.02.003

La Pray, M., & Ross, R. (1969). The graded word list: Quick gauge of reading ability. Journal of Reading, 12, 305-307.

Llerena, K., Park, S. G., McCarthy, J. M., Couture, S. M., Bennett, M. E., & Blanchard, J. J. (2013). The Motivation and Pleasure Scale-Self-Report (MAP-SR): Reliability and validity of a self-report measure of negative symptoms. Comprehensive Psychiatry, 54, 568-574. http://dx .doi.org/10.1016/j.comppsych.2012.12.001

Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. American Psychologist, 57, 705-717. http://dx.doi.Org/10.1037/0003-066X.57.9 .705

Lund, A. M. (2001). Measuring usability with the USE Questionnaire^. Usability Interface, 8, 3-6.

Luther, L., Firmin, R. L., Lysaker, P. H., Minor, K. S., & Salyers, M. P. (2018). A meta-analytic review of self-reported, clinician-rated, and performance-based motivation measures in schizophrenia: Are we measuring the same “stuff”? Clinical Psychology Re-view, 61, 24-37. http:// dx.doi.org/10.1016/j.cpr.2018.04.001

Luther, L., Firmin, R. L., Vohs, J. L., Buck, K. D., Rand, K. L., & Lysaker, P. H. (2016). Intrinsic motivation as a mediator between metacognition deficits and impaired functioning in psychosis. British Journal of Clinical Psychology, 55, 332-347. http://dx.doi.org/10.1111/bjc.12104

Luther, L., Fischer, M. W., Firmin, R. L., & Salyers, M. P. (2019). Clarifying the overlap between motivation and negative symptom measures in schizophrenia research: A meta-analysis. Schizophrenia Research, 206, 27-36. http://dx.doi.org/10.1016/j.schres.2018.10.010

Mas-Exposito, L., Amador-Campos, J.A., Gomez-Benito, J.,& Lalucat-Jo, L., & the Research Group on Severe Mental Disorders. (2011). The World Health Organization Quality of Life Scale Brief Version: A validation study in patients with schizophrenia. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation, 20, 1079-1089. http://dx.doi.org/10.1007/s11136-011-9847-1

McCarthy, J. M., Treadway, M. T., Bennett, M. E., & Blanchard, J. J. (2016). Inefficient effort allocation and negative symptoms in individuals with schizophrenia. Schizophrenia Research, 170, 278 -284. http:// dx.doi.org/10.1016/j.schres.2015.12.017

Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., . . . Wood, C. E. (2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine, 46, 81-95. http://dx.doi.org/10 .1007/s12160-013-9486-6

Montes, J. M., Medina, E., Gomez-Beneyto, M., & Maurino, J. (2012). A short message service (SMS)-based strategy for enhancing adherence to antipsychotic medication in schizophrenia. Psychiatry Research, 200, 89-95. http://dx.doi.org/10.1016/j.psychres.2012.07.034

Myerson, J., Baumann, A. A., & Green, L. (2014). Discounting of delayed rewards: (A)theoretical interpretation of the Kirby questionnaire. Behavioural Processes, 107, 99-105. http://dx.doi.org/10.1016/j.beproc.2014 .07.021

Nakagami, E., Xie, B., Hoe, M., & Brekke, J. S. (2008). Intrinsic motivation, neurocognition and psychosocial functioning in schizophrenia: Testing mediator and moderator effects. Schizophrenia Research, 105, 95-104. http://dx.doi.org/10.1016/j.schres.2008.06.015

Naslund, J. A., Aschbrenner, K. A., & Bartels, S. J. (2016). How people with serious mental illness use smartphones, mobile apps, and social media. Psychiatric Rehabilitation Journal, 39, 364-367. http://dx.doi .org/10.1037/prj0000207

Naslund, J. A., Marsch, L. A., McHugo, G. J., & Bartels, S. J. (2015). Emerging mHealth and eHealth interventions for serious mental illness: A review of the literature. Journal of Mental Health (Abingdon, England), 24, 321-332. http://dx.doi.org/10.3109/09638237.2015.1019054

Nyaruka. (2016). TextIt. Retrieved from https://textit.in/

Payne, H. E., Lister, C., West, J. H., & Bernhardt, J. M. (2015). Behavioral functionality of mobile apps in health interventions: A systematic review of the literature. JMIR mHealth and uHealth, 3, e20. http://dx.doi.org/ 10.2196/mhealth.3335

Pijnenborg, G. H., Withaar, F. K., Brouwer, W. H., Timmerman, M. E., van den Bosch, R. J., & Evans, J. J. (2010). The efficacy of SMS text messages to compensate for the effects of cognitive impairments in schizophrenia. British Journal of Clinical Psychology, 49, 259 -274. http://dx.doi.org/10.1348/014466509X467828

Reddy, L. F., Horan, W. P., Barch, D. M., Buchanan, R. W., Dunayevich, E., Gold, J. M., . . . Green, M. F. (2015). Effort-based decision-making paradigms for clinical trials in schizophrenia: Part 1—Psychometric characteristics of 5 paradigms. Schizophrenia Bulletin, 41, 1045-1054. http://dx.doi.org/10.1093/schbul/sbv089

Rotondi, A. J., Eack, S. M., Hanusa, B. H., Spring, M. B., & Haas, G. L. (2015). Critical design elements of e-health applications for users with severe mental illness: Singular focus, simple architecture, prominent contents, explicit navigation, and inclusive hyperlinks. Schizophrenia Bulletin, 41, 440-448. http://dx.doi.org/10.1093/schbul/sbt194

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. http://dx.doi.org/10.1037/0003-066X.55.1.68

Salamone, J. D., Cousins, M. S., McCullough, L. D., Carriero, D. L., & Berkowitz, R. J. (1994). Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not free food consumption. Pharmacology, Biochemistry and Behavior, 49, 25-31. http://dx .doi.org/10.1016/0091-3057(94)90452-9

Schlosser, D. A., Campellone, T. R., Truong, B., Etter, K., Vergani, S., Komaiko, K., & Vinogradov, S. (2018). Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophrenia Bulletin, 44, 1010-1020. http://dx.doi.org/ 10.1093/schbul/sby078

Schut, H. A., & Stam, H. J. (1994). Goals in rehabilitation teamwork. Disability and Rehabilitation, 16, 223-226. http://dx.doi.org/10.3109/ 09638289409166616

Skevington, S. M., Lotfy, M., & O'Connell, K. A., & the WHOQOL Group. (2004). The World Health Organization's WHOQOL-BREF quality of life assessment: Psychometric properties and results of the international field trial. A report from the WHOQOL group. Quality of Life Research: An International Journal of Quality of Life Aspects of Prealmenl, Care & Rehabilitation., 13, 299-310. http://dx.doi.org/10 .1023/B:QURE.0000018486.91360.00

Spaniel, F., Vohlidka, P., Kozeny, J., Novak, T., Hrdlicka, J., Motlova, L., . . . Hoschl, C. (2008). The Information Technology Aided Relapse Prevention Programme in Schizophrenia: An extension of a mirrordesign follow-up. International Journal of Clinical Practice, 62, 19431946. http://dx.doi.org/10.1111/j.1742-1241.2008.01903.x

Strauss, G. P., Waltz, J. A., & Gold, J. M. (2014). A review of reward processing and motivational impairment in schizophrenia. Schizophrenia Bulletin, 40, S107-S116. http://dx.doi.org/10.1093/schbul/sbt197

Strauss, J. S., & Carpenter, W. T., Jr. (1977). Prediction of outcome in schizophrenia. III. Five-year outcome and its predictors. Archives of General Psychiatry, 34, 159-163. http://dx.doi.org/10.1001/archpsyc .1977.01770140049005

Treadway, M. T., Buckholtz, J. W., Schwartzman, A. N., Lambert, W. E., & Zald, D. H. (2009). Worth the ‘EEfRT'? The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS ONE, 4, e6598. http://dx.doi.org/10.1371/journal.pone.0006598

Treadway, M. T., Peterman, J. S., Zald, D. H., & Park, S. (2015). Impaired effort allocation in patients with schizophrenia. Schizophrenia Research, 161, 382-385. http://dx.doi.org/10.1016/j.schres.2014.11.024

United States Department of Health and Human Services. (1998). E9: Statistical Principles for Clinical Trials, recommended for adoption to the regulatory bodies of the European Union. Washington, DC: Center for Drug Evaluation and Research.

Webb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010). Using the internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. Journal of Medical Internet Research, 12, e4. http://dx.doi.org/10.2196/jmir.1376

Whitehead, A. L., Julious, S. A., Cooper, C. L., & Campbell, M. J. (2016). Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Statistical Methods in Medical Research, 25, 1057-1073. http://dx.doi.org/10.1177/0962280215588241

Woods, S. W. (2003). Chlorpromazine equivalent doses for the newer atypical antipsychotics. The Journal of Clinical Psychiatry, 64, 663667. http://dx.doi.org/10.4088/JCP.v64n0607

Woods, S. W. (2011). Chlorpromazine equivalent doses for atypical antipsychotics: An update. Retrieved from http://scottwilliamwoods.com/ equivalencesupdate.php

Yanos, P. T., Roe, D., West, M. L., Smith, S. M., & Lysaker, P. H. (2012). Group-based treatment for internalized stigma among persons with severe mental illness: Findings from a randomized controlled trial. Psychological Services, 9, 248-258. http://dx.doi.org/10.1037/a0028048 Yu, L. Q., Lee, S., Katchmar, N., Satterthwaite, T. D., Kable, J. W., &

Wolf, D. H. (2017). Steeper discounting of delayed rewards in schizophrenia but not first-degree relatives. Psychiatry Research, 252, 303309. http://dx.doi.org/10.1016/j.psychres.2017.02.062

Yu, R. (2012). Regional white matter volumes correlate with delay discounting.

PLoS ONE, 7, e32595. http://dx.doi.org/10.1371/journal.pone.0032595

Received December 4, 2019

Revision received May 29, 2020

Accepted June 5, 2020 ■

Members of Underrepresented Groups: Reviewers for Journal Manuscripts Wanted

If you are interested in reviewing manuscripts for APA journals, the APA Publications and Communications Board would like to invite your participation. Manuscript reviewers are vital to the publications process. As a reviewer, you would gain valuable experience in publishing. The P&C Board is particularly interested in encouraging members of underrepresented groups to participate more in this process.

If you are interested in reviewing manuscripts, please write APA Journals at Reviewers@apa.org. Please note the following important points:

APA now has an online video course that provides guidance in reviewing manuscripts. To learn more about the course and to access the video, visit http://www.apa.org/pubs/journals/resources/ review-manuscript-ce-video.aspx.

Psychiatric Quarterly

https://doi.org/10.1007/s11126-020-09792-9

ORIGINAL PAPER


Development and Initial Testing of an mHealth Transitions of Care Intervention for Adults with Schizophrenia-Spectrum Disorders Immediately Following a Psychiatric Hospitalization

Ethan Moitra1© Hyun Seon Park2 Brandon A. Gaudiano1,2,3

Published online: 02 July 2020

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Abstract

An important period in the care of patients with schizophrenia-spectrum disorders is when they transition from inpatient to outpatient services and are at increased risk for relapse and rehospitalization. Thus, we developed and examined the initial feasibility, acceptability, and clinical effects of an mHealth transitions of care intervention (Mobile AfterCare Support; MACS) in an open trial. Ten adults with schizophrenia-spectrum disorders were recruited during their index psychiatric hospitalization and enrolled prior to discharge. Measures of feasibility, acceptability, and MACS targets were administered at baseline and a 1-month follow-up. Drawing on skills from Cognitive Behavioral Therapy for Psychosis (CBTp), MACS delivered brief assessments of clinically relevant variables, followed by just-in-time interventions for patients starting immediately post-discharge. Individuals completed about one session per day on average as expected. Overall, measures of MACS usability and satisfaction were positive. T-test analyses showed that dysfunctional coping strategies significantly decreased from baseline to 1-month followup. Results also revealed statistically significant reductions in psychiatric symptoms over 1-month follow-up. This study demonstrates the feasibility and acceptability of MACS, a new app-based intervention targeting transitions of care for patients with psychosis. The field is turning to the use of mobile technology as a means of augmenting service delivery and providing real-time assessment and intervention for patients at risk. MACS is a promising adjunctive intervention that warrants further testing in a randomized controlled trial.

Keywords Mobile technology. Psychiatric hospitalization. Psychosis . Schizophrenia. Treatment adherence. Cognitive behavior therapy for psychosis

X Ethan Moitra ethan_moitra@brown.edu

Extended author information available on the last page of the article

Introduction

The global burden of schizophrenia-spectrum disorders is substantial, with estimated treatment costs exceeding $100 billion per year [1, 2]. Indeed, psychosis is among the top 25 causes of disability worldwide [3]. Schizophrenia-spectrum disorders are associated with significant functional impairments and high rates of relapse [4-6], often making treatment complicated and costly. These challenges are particularly salient when individuals with psychosis transition from inpatient to outpatient care. Compared with more stable outpatients, individuals with psychosis post-hospitalization often have more cognitive impairments, problems with treatment connection/engagement, housing insecurity, medication side effects, and suicidality [7]. Recent psychiatric hospitalization predicts treatment nonadherence [8-10], and the transition from inpatient to outpatient services is associated with increased stress and premature treatment drop out [7, 11]. Despite the post-discharge period being a time of elevated risk, clinical settings often lack feasible and effective services to support patientsreturn to the community. Moreover, there is no gold standard intervention and delivery method that would be recommended to support treatment adherence and effective coping among these individuals post-discharge, suggesting the need for further research in this area.

Cognitive Behavioral Therapy for Psychosis (CBTp)

To date, adherence interventions for psychosis have produced promising, albeit mixed results [12-14]. The most fruitful approaches have drawn on cognitive behavioral therapy for psychosis (CBTp) [13-15]. CBTp teaches self-coping with illness by fostering active, planned, and effective problem solving to alleviate distress and improve functioning [16]. Another key focus of CBTp is to support treatment engagement and medication/ appointment adherence, which are essential to the management of schizophrenia [17]. In addition to research showing that CBTp leads to decreased symptoms and improved functioning beyond medications alone [18], studies show that CBTp is effective at improving coping with illness [19]. However, more work is needed to determine how to best deliver these interventions, particularly to help patients during the post-hospitalization period.

Digital Mental Health Services for Adults with Psychosis

One promising pathway to efficiently support patients with psychosis as they return to the community following hospitalization might be through the delivery of CBTp interventions using digital mental health or mobile health (mHealth) services. mHealth, often rooted in ecological momentary assessment and intervention [20-22], refers to technology-based service protocols that can improve patientsfunctioning and/or reduce symptoms [23]. As shown in recent systematic reviews [24, 25], there is a growing body of research into digital mental health services for patients with psychosis, including studies supporting their feasibility, acceptability, and efficacy.

In terms of acceptability, qualitative feedback has shown that community-based patients with psychosis endorse the benefits of integrating mobile technology into clinical care, including improving patient-provider communication, as well as presenting an opportunity to trigger early intervention [26]. Moreover, studies show that people with schizophrenia largely accept and can successfully complete mobile device-based assessment, with compliance rates typically comparable to those of nonclinical populations [27-33]. Further, moderately high engagement (44%) with mHealth services has been observed for up to 6 months in a sample of individuals with psychosis [34].

mHealth interventions using CBTp principles for individuals with psychosis also show promising efficacy. For example, in a pilot trial of a text messaging-based intervention for community-dwelling adults with psychosis, results showed that the digital intervention lead to improved medication adherence, increased social interactions, and reduced severity of auditory hallucinations [28]. Another study showed that text messaging helped individuals with psychosis to achieve their goals, such as attending outpatient treatment appointments or completing activities of daily living [35]. In a study of community-based individuals, a smartphone-based intervention resulted in reductions in depression, general psychopathology, and psychotic symptoms [36]. Most recently, a study that blended brief mobile intervention and face-to-face coping-focused therapy showed improved coping and reduced severity of auditory hallucinations [37].

In sum, data show that mental health interventions delivered through mobile devices are feasible and acceptable to patients with psychosis. Moreover, studies show that these interventions can produce significant reductions in symptoms and improve treatment adherence. However, most studies used relatively stable, community-based samples with psychosis. One study [34] focused on recently hospitalized individuals, but inclusion criteria were quite broad, allowing for hospital discharge to have occurred within the past 60 days, and patients were not transitioned directly from inpatient to outpatient care. Thus, the field is lacking systematic research focused on patients recruited during an index hospitalization and followed immediately post-discharge. This is an ideal period to leverage digital technology to deliver CBTp-based care, given that we know that these patients are at increased risk for treatment nonadherence and drop out, continued functional impairment, and the re-emergence of significant psychopathology shortly after they leave the hospital.

Rationale for the Present Study

The current study was a treatment development project to create a new mHealth intervention for patients with psychosis during the transition from inpatient to outpatient care. Thus, our aim was to examine the initial feasibility, acceptability, and possible effects of the newly developed mobile intervention (Mobile After-Care Support; MACS) for patients with psychosis post-hospitalization. MACS was developed as an application (app”) to be used on participantssmartphones. Using CBTp-based strategies, the app was designed to monitor patientstreatment adherence and symptoms and to intervene by providing brief, just-in-time interventions to support treatment adherence and participantsuse of healthy coping skills to manage their illness. We explored if MACS would be feasible, measured by participantsfamiliarity with and ability to use mobile devices, willingness to participate and remain in the study, and successful navigation of problems or issues encountered when using MACS. We also examined MACSs acceptability, characterized by app engagement rates, ratings of usability and satisfaction, and qualitative feedback about patientsexperiences. Lastly, we examined directional changes in MACSs intervention targets and outcomes to determine initial target engagement for testing in a future randomized controlled trial.

Material and Methods

Participants

Ten adults with schizophrenia-spectrum disorders participated in this study. Inclusion criteria were: (a) currently hospitalized (inpatient psychiatric facility); (b) diagnosed with DSM-5 criteria for schizophrenia or schizoaffective disorder based on the Structured Clinical Interview for DSM-5 (SCID-5; [38]); (c) 18 years or older; (d) prescribed oral antipsychotic medication upon discharge; and (e) able to speak and read English (materials written at a 5th grade reading level). Exclusion criteria were: (a) alcohol/drug use disorders at moderate or severe level based on SCID (mild substance use disorders were permitted); (b) planned discharge to supervised living setting or participation in formal outpatient adherence programs in which patients did not control their mediation administration (e.g., medication packaging); or (c) pregnant or had a medical condition contraindicating use of antipsychotic medications (e.g., dementia as indicated by patientsmedical charts). See Table 1 for summary of demographic and clinical characteristics of the sample.

Procedures

Recruitment occurred during participantsindex inpatient admission at a private, acute-care psychiatric hospital in the northeast region of the U.S. The study was approved by the Institutional Review Board of the hospital. Electronic medical records for newly admitted patients were screened after obtaining a Protected Health Information waiver for this purpose. Participants completed a baseline assessment prior to hospital discharge to confirm eligibility and were then asked to complete a 1-month follow-up. Additional follow-ups were conducted for pilot purposes, but not examined here as 1-month was considered the target period for MACS treatment.

Table 1 Baseline demographic characteristics (n = 10)

n (%)

Mean (± SD)

Age

44.4 (±13.9)

Gender (Female)

6 (60.0%)

Race

5 (50.0%)

White

2 (20.0%)

African American/Black

3 (30.0%)

Multiple races

Ethnicity (Latinx)

1 (10.0%)

Single (never married)

5 (50.0%)

Education (years)

13.4 (±1.9)

Household income (< $40,000/yr.)

7 (87.5%)

Full- or part-time employment

4 (40.0%)

Physical and/or psychiatric disability

3 (30.0%)

Retired

1 (10.0%)

Unemployed

2 (20.0%)

Primary Diagnosis

Schizophrenia

6 (60.0%)

Schizoaffective disorder (Bipolar)

3 (30.0%)

Schizoaffective disorder (Depressive)

1 (10.0%)

Missing income data: n = 2

Follow-ups occurred in person whenever possible, but some (n = 6) were conducted remotely for participantsconvenience. Research assistants were trained to initial interrater reliability (kappa > .80) on the interview-administered measures, with periodic checks to prevent against drift.Participants were compensated $30 for each assessment (baseline and 1-month). The MACS app was either downloaded onto the participants smartphone or if needed, a study device was provided with the app pre-loaded (n = 4; 40%). All participants practiced responding to MACS app sessions to familiarize themselves with the program and troubleshoot technical problems prior to discharge.

Mobile After-Care Support (MACS) App

The MACS app was programmed using an established mobile software service (ilumivu.com), which provided a secure, HIPAA-compliant application (Android or Apple IOS compatible). The protocol consisted of 3 randomly scheduled prompts during daytime hours (9 am - 9 pm). Additionally, users could initiate a MACS session on demand.Each session was designed to take 5-10 min to complete. Sessions began with brief assessments about coping, substance use, symptoms, treatment adherence, behavioral activation, and quality of life. Based on responses to these initial questions, participants were then prompted with individualized intervention skills. After obtaining releases of information from the individual, research reports, containing summarized MACS data, were sent to participantsoutpatient providers at baseline, two weeks later, and at the conclusion of the 1-month period. These reports explained that the individual was participating in the MACS study and summarized app-collected data related to symptoms endorsed.

Because MACS was designed as a mobile self-management intervention, we chose to focus on techniques that taught participants active coping strategies to manage illness-related distress and to foster adherence to medications and treatment appointments, given the key role these factors play in preventing relapse and enhancing long-term recovery. Primarily, MACS was constructed from common components adapted from CBTp studies, including those testing mobile interventions to improve self-coping and adherence behaviors [27, 28]. Any reported treatment nonadherence during the initial mobile assessment was prioritized as a topic in need of intervention via MACS. Participants were asked to choose from a variety of possible reasons for nonadherence. If the reported reason was primarily logistical (e.g., ran out of pills), participants were instructed to contact their provider at the community clinic to address the problem. This information was also conveyed to providers through the periodic reports so that the community clinic could reach out to participants to address adherence issues. If nonadherence was attributed to medication concerns (e.g., does not believe medications help), MACS used CBTp techniques that encouraged participants to communicate concerns to providers, reminded them of costs vs. benefits of medication in terms of symptom management, and taught them to engage in other brief problem-solving strategies delivered through the app [39]. If appointment nonattendance was reported, similar problem-solving strategies were suggested. The MACS app also provided interactive exercises designed to teach participants coping skill using CBTp exercises (e.g., Is there another explanation for what is going on right now? Lets explore some examples.or Try doing what you want despite what the voices say. Lets practice how to do this now.). Other domains that MACS targeted through CBTp interventions included: lack of social engagement/support, negative affect, low life satisfaction, and substance abuse.

Measures of Feasibility and Acceptability

We examined feasibility by assessing participantsmobile device use and connectivity using a study-designed phone usage questionnaire. Participantsneed for additional MACS training or trouble-shooting during use of the app was cataloged to further quantify feasibility. In an exit interview at the 1-month follow-up, we asked participants about positive and negative aspects of using the app and how it affected them. Lastly, the following self-report measures were used to further assess feasibility and acceptability at the 1-month follow-up:

Client Satisfaction Questionnnaire-8 (CSQ-8; [40]). The CSQ-8 is an 8-item reliable and valid measure designed to assess individualssatisfaction with services or an intervention. Higher scores indicated greater satisfaction. The CSQ-8 consistently shows high reliability (e.g., a =.93; [41].

System Usability Scale (SUS; [42]). Initially developed to examine usability of products or services, the SUS is a reliable and valid 10-item self-report that was administered to specifically examine the usability of the MACS app. Example items included, I would imagine that most people would learn to use this app very quickly.In a study using a large collection of data and usability ratings [43], the SUS showed good reliability (a = .85).

The Usefulness, Satisfaction, and Ease of Use Questionnaire (USE; [44]). The USE is a 30-item self-report measure found to be reliable and valid for assessing products or services. It examines four dimensions of usability related to the MACS app: (a) usefulness;

Adherence, Coping, Functioning, and Symptom Measures

At baseline and the 1-month follow-up, the following assessment measures were administered:

Antipsychotic Medication Beliefs and Attitudes Scale (AMBAS; [46]). The AMBAS is a reliable and valid 12-item self-report that shows initial evidence of reliability and validity. It assesses medication beliefs and attitudes, including factors related to shame and stigma. Higher scores mean more positive medication beliefs.

Brief Adherence Rating Scale (BARS; [47]). The BARS is an interviewer administered measure assessing the percentage of antipsychotic medication doses taken vs. prescribed over the past month. The BARS has shown good reliability (a =.92) when administered to individuals with psychosis [47].

Brief Coping Orientation to Problems Experienced (Brief COPE; [48]). The Brief COPE is a 28-item self-report measure of various coping approaches, include problematic or maladaptive approaches. In this study, we used the three subscales constructed by Coolidge and colleagues [49]: Emotion-focused coping strategies, Problem-focused strategies, and Dysfunctional coping strategies. The Brief COPE shows good internal consistency [48].

Brief Psychiatric Rating Scale-18-Item (BPRS; [50]). The BPRS is an interviewer-rated measure of overall psychiatric symptoms, including anxiety, depression, and psychosis. It shows good validity for distinguishing symptoms associated with psychosis [51].

World Health Organization Disability Assessment Schedule 2.0 (WHODAS; [52]). The WHODAS is a self-report measure of functional impairment. It probes for impairment in activities of daily living, cognition, mobility, self-care, and socialization. Higher scores indicate greater functional impairment. In a sample of adults with schizophrenia, the WHODAS showed high internal consistency and validity [53].

Statistical Analyses

All statistical analyses were conducted in SPSS. We calculated descriptive statistics for measures of acceptability and feasibility. Bivariate correlational analyses were used to examine baseline demographic and clinical variables associated with MACS engagement. Within-subjects t-tests, as well as related effect sizes (Cohens d) and confidence intervals, were used to compare baseline to 1-month time points regarding outcomes. Summaries of exit interviews were compiled to illustrate pros and cons of MACS.

Results

Study Recruitment and Retention

See Fig. 1 for CONSORT diagram depicting participant flow. Of the 15 patients approached for the study, 10 (66.7%) agreed to participate in the study. All 10 of these individuals completed the baseline assessment and were given the MACS app. Seven participants (70%) completed all measures administered at the 1-month follow-up and one additional participant partially completed 1-month measures. During the intervention, one participant (10%) was rehospitalized.

Personal Technology Access/Usage

At baseline, four participants (40%) did not own a smartphone (and were given a device to use during the study) and six participants (60%) did. A majority of participants reported using a mobile phone for >1 h/day (n = 7; 70%). Access to the internet and Wi-Fi were similarly high (n = 9; 90%), although very few participants had an e-mail address (n = 2; 20%).

MACS Feasibility and Completion

Participants were instructed to contact MACS staff to troubleshoot technical difficulties. Additionally, if non-usage of the app was observed for several days, study staff contacted the participant to determine the reason for this and address any issues. Some participants reported that the session did not show up correctly in the app and that the person was having difficulty navigating through the app, likely due to lack of technology fluency. Most issues were resolved quickly with minimal support.

A total of 275 MACS sessions were completed by participants. A majority of these sessions were completed when the participant was at home (78.9%). One participant did not complete

any MACS sessions. Among the other nine participants, total engagement with the MACS app reflected, on average, approximately one session per day (M = 28.0 sessions, SD = 28.6). Incomplete (i.e., started, but not finished) MACS sessions were rare (M = 2.6 sessions, SD = 5.2). In correlational analyses of baseline demographics and other clinical variables in relation to MACS completion rates, results revealed that baseline BPRS score was significantly correlated with MACS completion rates, such that increased psychiatric severity was associated with increased completion rates (r = .73, p = .025). Otherwise, none of the these variables significantly correlated with MACS engagement (ps > .05).

MACS Usability and Acceptability Ratings

Usability was measured with the CSQ-8, USE, and SUS measures at the conclusion of the 1month MACS period. Mean item ratings for the CSQ-8 were above the midpoint on the 4-point scale (M = 2.4, SD = 0.2), suggesting overall positive satisfaction with MACS. Total SUS scores were also positive, with mean ratings of 75.4 (SD = 31.8). Scores >68 represent above averageusability [42]. Overall USE ratings in each subscale trended to the positive range as well. See Table 2 for a summary of participantssatisfaction ratings.

Participants chose to engage with a wide variety of MACS-provided coping skill interventions. The most commonly chosen MACS coping skills to focus on during the sessions related to coping with emotions (30.7%) followed by behavioral activation (21.6%). Coping skills related to psychotic symptoms also were particularly well-received. See Table 3 for summary of coping skills engagement and participantssatisfaction with the chosen skill.

At the conclusion of the 1-month MACS intervention, we conducted exit interviews to better understand participantsexperiences using the app. Feedback was mixed, but mainly positive. Most participants reported that they found the app easy to useand appreciated how the app prompted them to think about how Im feeling by checking in.Some participants commented that the app had too many sessionsand that the content could sometimes be not as personable as receiving coping advice in-person or by phone.Participants suggested that MACS could be improved by adding a wider array of coping skills and providing further training or explanation about coping skills.

MACS Targets and Clinical Outcomes

Reported antipsychotic medication nonadherence and outpatient treatment nonadherence were minimal (4.7% and 3.3%, respectively). Only one participant noted having missed a treatment appointment; otherwise, participants reported being 100% adherent to outpatient appointments. T-test analyses comparing baseline to 1-month data showed that use of dysfunctional coping strategies significantly decreased during the 1-month period using MACS: t(7) = 5.40, p =.002, d =1.45. Results also revealed a statistically significant reduction in psychiatric symptoms, as measured by the BPRS: t(7) = 6.46, p = .002, d =2.13. Moreover, analyses supported expectations for directional improvements in other MACS outcomes, including improved functioning and increased positive medication beliefs, although these results were statistically nonsignificant. See Table 4 for summary of results.

Discussion

This study demonstrates the feasibility and acceptability of MACS, a new mHealth app-based intervention, designed to support treatment adherence and healthy coping among adults with psychosis immediately following hospital discharge. Despite being approached while

Table 2 MACS app acceptability ratings (n = 7)

Mean (± SD)

Range

CSQ-8 (by item)

2.4 (±0.2)

2.1-2.9

SUS (total score)

75.4 (±31.8)

15-100

USE - Ease of learning (by item)

5.0 (±2.5)

1-7

USE - Ease of use (by item)

4.8 (±2.2)

1-6.8

USE - Satisfaction (by item)

4.4 (±1.9)

1-6.3

USE - Usefulness (by item)

3.8 (±1.6)

1-6

CSQ-8 Client Satisfaction Questionnnaire-8, SUS System Usability Scale, USE The Usefulness, Satisfaction, and Ease of Use Questionnaire

Table 3 MACS app coping skills selected by participants at each session

Coping skill

n (%)

Skill Satisfaction (Mean (± SD))

Skill Satisfaction (Range)

Emotion management

74 (31.5%)

3.08 (±1.08)

1-5

Behavior activation

52 (22.1%)

3.64 (±1.04)

2-5

Quality of life enhancement

43 (18.3%)

3.34 (±1.08)

2-5

Psychosis management

40 (17.0%)

4.34 (±1.13)

1-5

Social support enhancement

20 (8.5%)

3.60 (±0.75)

2-5

Substance use change

6 (2.6%)

3.33 (±1.34)

2-5

hospitalized, a majority of patients agreed to participate in the study. The most highly rated coping skills offered by MACS were those related to managing symptoms of psychosis and facilitating behavioral activation. In addition, the most frequently chosen copings skills related to emotion regulation. This is likely reflective of the variety of symptoms individuals with psychosis continue to experience at hospital discharge, even if they are less acutely ill. Participant feedback was mostly positive as many noted the benefits of being prompted to reflect on their symptoms and functioning, as well as in receiving brief CBTp-based support. Some participants expressed a desire for MACSs coping skills to be more varied and that more support in using the coping skills would have been helpful. Otherwise, minimal troubleshooting was needed in which staff helped participants navigate MACS technical issues. Many of these problems were likely due to user error or low technology fluency, rather than problems with the app per se.

MACS completion rates were generally good, with individuals completing at least one session per day on average, which was the target rate for the study. Only one participant did not complete any MACS sessions. Correlational analyses showed that baseline psychiatric symptoms significantly predicted MACS session completion rates, with higher symptoms leading to higher MACS engagement. This relationship makes sense from a clinical perspective as individuals who might be acutely distressed could be more motivated to engage with MACS, particularly if they found it helpful. Although participants were prompted three times per day, responding to assessments and engaging in the interventions at least once daily far surpasses the weekly frequency of assessment and care provided on a typical outpatient schedule. Engagement with mHealth apps vary in this population, but published studies tend

Table 4 Comparison of baseline and 1-month outcomes (n = 7)

Mean (±SD)

Mean (±SD)

t-test

(p value)

Cohen’s d (95%CI)

BPRS

46.4 (±6.7)

30.4 (±8.5)

6.46 (<.001)

2.13 (.39, 3.87)

WHODAS

34.7% (±15.3%)

28.5% (±21.2%)

.83 (.444)

.34 (.74, 1.43)

BARS

96.9% (±8.8%)

97.5% (±4.6%)

-.16 (.875)

.06 (-.76, .88)

AMBAS

32.3 (±3.1)

33.8 (±8.4)

-.36 (.733)

.40 (-.71, 1.51)

COPE - Dysfunctional coping

26.9 (±5.3)

18.0 (±4.1)

5.40 (.002)

1.45 (.05, 2.96)

COPE - Emotion-focused strategies

24.4 (±7.6)

24.4 (±8.7)

<.01 (>.999)

0 (-.91, .91)

COPE - Problem-focused strategies

16.6 (±4.9)

14.6 (±6.8)

1.35 (.225)

.34 (.60, 1.31)

3 individuals did not complete the 1-month assessment. BARS Brief Adherence Rating Scale, BPRS Brief Psychiatric Rating scale-18-item, WHODAS World Health Organization Disability Assessment Schedule 2.0, AMBAS Antipsychotic Medication Beliefs and Attitudes Scale, COPE Brief Coping Orientation to Problems Experienced

to define adequate engagement as participants completing at least 20% of prompted sessions [33, 37, 54]. MACS is designed to be an adjunct to outpatient care, meaning that it will support other services, but should not fully replace more traditional treatment modalities.

Although this was a pilot study with a small sample, results suggested significant improvements in certain aspects of coping with illness and overall psychiatric symptoms. Adherence was high at baseline and remained high at follow-up. Inconsistent changes were observed for the other measures; although sample size was small and confidence intervals around effects were large. Overall, our findings extend prior mHealth work in psychosis (e.g., [29, 55]) by further demonstrating the feasibility, acceptability, and potential efficacy of mHealth interventions in adults with psychosis. By initiating MACS at hospital discharge, this study built on prior work by targeting a novel, yet high risk time period.

Limitations

This was a small sample, which might not be representative of other patients with psychosis. Due to the longitudinal nature of the mHealth intervention, participants with relatively stable housing at discharge and reliable access to telephone communications and transportation were prioritized for recruitment. Furthermore, the studys inclusion requirements of a SCID-5 diagnosis of schizoaffective disorder or schizophrenia excluded patients who were hospitalized with first psychotic episodes and those who were diagnosed with unspecified psychosis or schizophreniform disorder. In addition, the measures of medication adherence used were selfreport and designed for oral medications only, and as a result, patients who were only prescribed long acting injectable antipsychotics were excluded from the study. Finally, clinical improvements reported here cannot necessarily be attributed to the MACS intervention because of the lack of a control group. Further testing is needed to examine MACS in a randomized controlled design now that it has been shown to be feasible and acceptable.

Conclusions

The present study is the first to our knowledge to specifically target treatment adherence and coping among hospitalized individuals with psychosis immediately upon discharge, using an mHealth approach. Currently available interventions for improving coping and medication/ appointment adherence in psychosis have been shown to be efficacious, but are not routinely utilized in real-world clinical settings due to barriers related to feasibility, cost, and access. The field is turning to the use of mobile technology as a means of augmenting service delivery and providing real-time assessment and intervention in more efficient ways. However, there are notable gaps in existing research. Digital health interventions are a possible solution to these issues and they warrant further research.

References

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Ethan Moitra is an Assistant Professor at the Warren Alpert Medical School of Brown University. He received his doctoral degree from Drexel University.

Hyun Seon Park is research staff at Butler Hospital. She received her bachelor’s degree from the University of Illinois - Urbana - Champaign.

Brandon A. Gaudiano is an Associate Professor at the Warren Alpert Medical School of Brown University. He received his doctoral degree from Drexel University.

Affiliations

Ethan Moitra1 Hyun Seon Park2 Brandon A. Gaudiano1'2'3


Contents lists available at SciVerse ScienceDirect

Psychiatry Research

journal homepage: www.elsevier.com/locate/psychres


A short message service (SMS)-based strategy for enhancing adherence to antipsychotic medication in schizophrenia

Jose Manuel Montes a, Esteban Medina b, Manuel Gomez-Beneytoc, Jorge Maurino b,n

a Department of Psychiatry, Hospital Universitario del Sureste, CIBERSAM, Madrid, Spain

b AstraZeneca Medical Department, Madrid, Spain

c Department of Psychiatry, School of Medicine, University of Valencia, Valencia, Spain

ARTICLE INFO


ABSTRACT


Article history:

Received 18 January 2012

Received in revised form

8 June 2012

Accepted 24 July 2012


Background: The aim of this study was to assess the impact of a short message service (SMS)-based strategy on adherence to antipsychotic treatment.

Keywords:

Short message service (SMS)

Adherence

Schizophrenia

Antipsychotics

Cell phone


Methods: A multicentre, randomised, open-label, controlled, 6-month study with clinically stabilised outpatients with schizophrenia was conducted. The patients assigned to the intervention received daily SMS reminders to take their medication for 3 months. Self-reported medication adherence was determined using the Morisky Green Adherence Questionnaire (MAQ). Secondary outcomes were severity of illness, attitude towards medication, insight into illness and health-related quality of life. Results: A total of 254 patients were analysed. A significantly greater improvement in adherence was observed among patients receiving SMS text messages compared with the control group. The mean change in MAQ total score from baseline to month 3 was -1.0 (95% confidence interval (CI) -1.02, -0.98) and -0.7 (95%CI -0.72, -0.68), respectively (P=0.02). Greater improvement in negative, cognitive and global clinical symptoms at month 3 was observed. Attitude towards medication also significantly improved across the study in the intervention group versus the controls.

Conclusions: An SMS-based intervention seems feasible and acceptable for enhancing medication adherence. Further studies are needed to confirm whether this kind of intervention could be a complementary strategy to optimise adherence in schizophrenia.

© 2012 Elsevier Ireland Ltd. All rights reserved.

1. Introduction

Non-adherence to antipsychotic treatments is still an enormous challenge for clinicians in the management of patients with schizophrenia (Velligan et al., 2009). Partial and non-adherence to treatment are associated with poorer prognoses, higher risk of relapse and hospitalisation and suicide attempts (Ascher-Svanum et al., 2006; Leucht et al., 2006). The mean adherence rate with antipsychotic medications is 58%, with a range from 24% to 90% (Cramer and Rosenheck, 1998). Many factors contribute to partial and non-adherence in schizophrenia, including poor insight, a negative attitude towards medication, substance abuse and disorganisation (Lacro et al., 2002; Liu-Seifert et al., 2010).

Different strategies have been used to improve adherence to treatment, including patient and family psychoeducation, motivational interviewing, cognitive and behavioural approaches, assertive community models and other strategy combinations (Patterson et al., 2008; Julius et al., 2009; Goff et al., 2010; Barkhof

* Corresponding autor at: AstraZeneca Medical Department, Parque Norte. Edificio Roble, Serrano Galvache, 56. 28033 Madrid, Spain

E-mail address: jorgealejandro.maurino@astrazeneca.com (J. Maurino). et al., 2012). In addition, behavioural components such as reminders, self-monitoring tools, cues or reinforcements have proved to be useful, easy to implement and do not interfere with daily clinical practice (Dolder et al., 2003; Montes et al., 2011). Nevertheless, no specific intervention has demonstrated overwhelming success in improving adherence (Barkhof et al., 2012).

Internet and cell-phone services recently have been implemented to optimise communication between health-care providers and patients (Patrick et al., 2008). New technologies may be a powerful tool to be used as a method for reminding patients to take their medication (Wangberg et al., 2008; Borzekowski et al., 2009; Christensen and Hickie, 2010). Nowadays, mobile devices are extensively used among the population and have shown to be feasible and acceptable approaches for interventions in chronic physical illnesses, such as arterial hypertension, diabetes, malaria chemoprophylaxis or human immunodeficiency virus (HIV) antiretroviral treatment, as well as to improve compliance with hospital/outpatient appointments (Patrick et al., 2008; Smith et al., 2010). Cell phones incorporate a short message service (SMS) for sending text messages, constituting a cheap, low timeconsuming and discreet tool for communication (Patrick et al., 2008; Depp et al., 2010; Ehrenreich et al., 2011; Harrison et al., 2011; Makela et al., 2010; Van den Berg et al., 2011). SMS text messages used to remind patients with schizophrenia and bipolar disorder to take their medication have shown promising results in a non-randomised pilot study (Van Gent and Knoppert Van Der Klein, 2010). Pijnenborg et al. (2010) recently published a randomised trial to assess the efficacy of SMS messages in the cognitive rehabilitation of 62 patients with schizophrenia or related psychotic disorders. Patients achieved more of their goals in daily life. However, prompting with SMS did not lead to a significant increase in therapeutic adherence. Preliminary findings indicate that most patients are willing to use this method of communication and are able to do so with few problems (Spaniel et al., 2008Humphrey Beebe et al., 2010; Pijnenborg et al., 2010).

0165-1781/$-see front matter © 2012 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.psychres.2012.07.034


The primary aim of this study was to assess the efficacy of sending daily treatment reminders via SMS over 3 months for adherence to antipsychotic medication among outpatients with schizophrenia. We also assessed the impact on other outcome measures, including attitude towards medication, insight into illness, clinical severity and health-related quality of life.

2. Materials and methods

A prospective, randomised, open label, controlled, 6-month study was conducted in 56 outpatient psychiatric centres throughout Spain. These facilities were nominated for their good practice by researchers and experts in psychiatry community care. The study was approved by the institutional review board of the Fundacio Catalana dHospitals (Barcelona, Spain) (Clinical Trial Registry #NCT00873249). Data were collected from April 2009 to February 2010.

The criteria for patient recruitment were: 18-65 years of age, a diagnosis of schizophrenia according to the Diagnostic and Statistical Manual, fourth edition, text revision (DSM-IV-TR), clinically stable (operationalised as having had either any change in severity or new treatments initiated in the last 6 months), a single oral antipsychotic medication, follow-up as an outpatient, at least one affirmative answer (indicating suboptimal medication adherence) to the Morisky Green Adherence Questionnaire (MAQ) and availability of a cell phone capable of receiving SMS messages.

Those patients receiving long-acting injectable antipsychotic treatment were excluded.

After a complete description of the study to the participants, written informed consent was obtained. Patient confidentiality was maintained, since no identifying data were recorded in the study documentation.

Investigators included the first five consecutive patients that met the inclusion criteria for participating in the study. Competitive recruitment was established among centres. Randomisation codes were computer generated by our statistician and sealed in envelopes labelled with consecutive numbers. The envelopes were opened by the investigator in an ascending order and patients were allocated to the intervention or control group (CG).

2.2. SMS-based intervention

Participants assigned to the intervention received daily SMS reminders on their cell phones to take their medication for 3 months. Group assignment was based on a1:1 randomisation scheme. The SMS text received by the patients in the intervention group (SMSG) said: "Please remember to take your medication”. Patients could choose between receiving the message at either 11 a.m. or 2 p.m.

Patients assigned to the CG did not receive SMS messages or any other more intensive approach for increasing adherence than standard of care during the study period.

Between months 3 and 6, all patients were followed-up without receiving SMS messages.

An automated SMS messaging service integrated into a website was created ad-hoc for the study. Participating investigators needed a username and password to access the website and to include the patient cell-phone number and the date of starting the intervention. During each personal website visit, the investigator could check the current status of SMS reception for each patient.

2.3. Outcome measures

Assessments took place at baseline and at 3 and 6 months after randomisation. The primary outcome measure was the change from baseline to month 3 in the MAQ (Morisky et al., 1986) total score compared with the CG. The MAQ addresses how patients may fail to take medication as prescribed due to forgetfulness, carelessness, stopping the drug when they feel better or stopping treatment because they believe it makes them feel worse. It is a self-rated questionnaire consisting of four questions with yes/no answers. When the answer indicates a negative adherence issue, a score of 1 is recorded. The total score ranges from 0 (good adherence) to 4 (poor adherence). MAQ score at a threshold of 4I may be a valuable tool for identifying non-adherent patients in a cohort where adherence is low (Erickson et al., 2001). The scale has good levels of validity and reliability, and was initially developed to assess compliance in patients with arterial hypertension and occasionally in the context of psychiatric disorders (Gray et al., 2006).

Secondary outcome measures included the change from baseline to month 6 in the MAQ total score (3 months after stopping the SMS-based intervention). Clinical severity was assessed using the Clinical Global Impression - Schizophrenia scale (CGI-SCH) (Haro et al., 2003). The CGI-SCH consists of two categories: severity of illness (CGI-SCH-SI) and degree of change (CGI-SI-DC). The SCH category evaluates the situation during the week prior to the assessment, while the DC category evaluates the change from the previous evaluation (or from the phase preceding the trial). Each category contains five different ratings (positive, negative, depressive, cognitive and global), which are evaluated using a 7-point ordinal scale.

Attitude towards medication was assessed using the Spanish adaptation of the 10-item Drug Attitude Inventory (DAI-10) (Robles et al., 2004). The DAI-10 is a self-rated scale developed to measure subjective responses and attitudes of patients with chronic schizophrenic towards maintenance antipsychotic treatment. A positive total score means a positive subjective attitude (Hogan et al., 1992). Insight was measured using the first three items of the Scale to Assess Unawareness of Mental Disorder (SUMD) (Amador et al., 1994; Ruiz et al., 2008). These items assess subject general insight into having a mental disorder, the effects of medication upon the disorder and general understanding of the consequences of the disorder, respectively. The items are rated on a 5-point Likert scale (1 = ‘aware’ to 5=‘unaware’), with higher scores indicating poorer awareness. Health-related quality of life was assessed using the second part of the Spanish version of the EuroQol (EQ-5D) (Badia et al., 1999). This is a selfadministered instrument with proven validity for assessing quality-of-life differences in patients with schizophrenia of different degrees of severity (Konig et al., 2007). EQ-5D part two is a visual analogue scale (VAS) ranging from 0 (‘worst possible state of health’) to 100 (‘best possible state of health’).

A sample size of 286 patients was sought to detect differences in a mean MAQ total score of 0.5 between baseline and 3-6-month visits, achieving 80% of statistical power and assuming a standard deviation of 1.5 with a two-sided alpha level of 0.05 (Gray et al., 2006). With an estimated 20% attrition rate, this required a recruitment plan of 360 patients (180 in the intervention and 180 in the CG) at baseline.

To ensure that each participant in the SMS group was properly exposed to the intervention, our study protocol established a cut-off point according to opening or not of the SMS text messages by the patient. We hypothesised that a certain number of patients could have problems interacting with mobile phones due to poor motivation, disorganised daily living activities, etc. Patients with more than 7 consecutive days without properly receiving the SMS reminders on their cell phones were classified as ‘not properly exposed’ to the intervention, and excluded from the analysis.

Descriptive statistics were used to present patient demographics and clinical information. Independent samples t-tests and chi-square analyses were used to compare characteristics between the two study groups. To analyse the changes during the study, a ‘last observation carried forward’ (LOCF) approach was used, including those patients with a baseline evaluation and at least one posterior evaluation. A two-tailed significance level of 95% was considered for all analyses.

A stepwise linear regression model was constructed including all variables that were significant (p < 0.2) in the bivariate analysis to evaluate the association of other covariates in the improvement of adherence as measured by the changes in MAQ total score between visits. All statistical analyses were performed using Statistical Analysis Software (SAS) v.8.02 (SAS Institute Inc., Cary, NC, USA).

3. Results

3.1. Baseline characteristics

A total of 340 patients were included in the study. Twenty patients were excluded from the analysis due to major protocol deviations. An additional group of 66 patients were classified as ‘not properly exposed’ to the intervention and also excluded.

The analysis sample population comprised a total of 254 patients (100 in the SMSG and 154 in the CG). One patient in the SMSG and two in the CG dropped out of the study due to failure to comply with follow-up. The disposition of patients enrolled in this study is illustrated in Fig. 1.

At baseline, there were no substantial socio-demographic or major clinical differences between the control and intervention groups (Table 1). Baseline measures of treatment adherence, insight into illness, attitude towards medication, clinical severity and health-related quality of life were similar in both groups (Table 2). A trend towards lower level of insight (P=0.058) and a higher level of severity in positive and global symptoms according to the CGI-SCH SI score (P=0.096 and 0.072, respectively) was observed in the CG.

3.2. Primary outcome measure

A significantly greater reduction in MAQ total score was observed among patients receiving SMS reminders compared with the CG at month 3 (25% vs. 17.5%, respectively). The mean changes in MAQ total score as assessed by the LOCF analysis from baseline to month 3 were -1.0 (95%CI -1.02, - 0.98) and - 0.7 (95%CI -0.72, -0.68) in the SMSG and CG, respectively (P = 0.02) Table 3.

Thirty-seven patients (37%) achieved a mean MAQ total score < 1 at month 3 in the SMSG compared with 35 (22.7%) in the CG.

3.3. Secondary analysis

At the end of month 6 of follow-up, improvement in adherence was maintained: mean MAQ total score changes from baseline were-1.1 (95%CI -1.12, -1.08) and -0.8 (95%CI - 0.81, -0.78) in the SMSG and CG, respectively (P = 0.04) Table 3.

Most affirmative answers in MAQ were in items 1 and 4 (‘‘Do you ever forget to take your medicine?” and ‘‘Sometimes if you feel worse when you take the medicine, do you stop taking it?”, respectively). After 3 months of intervention, all MAQ items experienced a reduction of affirmative responses. Nevertheless, the improvement was significantly greater only in item 4: percentage of affirmative responses of 45% in the SMSG versus 59.1% in CG; P= 0.027 Table 4.

Improvement in depressive symptoms was similar in the two groups at the end of month 6 period, but mean reductions in the negative symptoms subscale of the CGI-SCH-SI were significantly higher for SMSG patients (-0.6, 95%CI -0.62, -0.58) than for CG patients (- 0.3, 95%CI -0.32, -0.28), P=0.03. No differences were observed in the other subscales of the CGI-SCH-SI at months 3 or 6. There was a significantly greater improvement at month 3 in negative (3.3 vs. 3.5), cognitive (3.3 vs. 3.6) and global symptoms (3.2 vs. 3.5) as assessed by the CGI-SCH-DG in the SMSG than in the CG, respectively (P < 0.05). A trend towards greater improvement in depressive symptoms was also observed (3.3 vs. 3.5; P=0.07). These differences could not be detected after month 6 of follow-up.

Patient subjective attitude towards medication significantly improved after 3 months of intervention (mean increase from baseline of 2.0, 95%CI 1.94, 2.06) versus those under conventional care (0.4, 95%CI 0.35, 0.45), P = 0.0003. This improvement continued after the end of the 6 months of follow-up (mean increases in DAI-10 score from baseline of 2.3, 95%CI 2.24, 2.36 and 0.9, 95%CI 0.85, 0.95 in the SMSG and the CG, respectively; P=0.002).

No significant changes were detected in awareness of illness at months 3 or 6.

After 3 months of intervention, improvement in quality of life was significantly greater in the SMSG (mean changes from baseline EQ-5D visual analogue scale (VAS) of 6.6, 95%CI 6.38, 6.82) compared to the CG (3.1, 95%CI 2.91, 3.29), P = 0.03. However, this improvement was similar in both groups at the end of follow-up.

Stepwise linear regression analyses at 3 and 6 months with the change in mean MAQ total score as the dependent variable is shown in Table 5. Factors significantly associated with a greater reduction in mean MAQ score after 3 months of intervention were receiving SMS reminders and a higher mean baseline DAI-10 score. Conversely, a larger number of hospitalisations, higher mean baseline MAQ scores, at least one previous hospitalisation and female gender were significantly associated with lower decreases in mean MAQ scores after 3 months of intervention. At month 6, the only factor associated with greater reductions in mean MAQ total scores was a higher baseline mean DAI-10 score, in which the effect was small.


Table 1

Baseline socio-demographic and illness-related characteristics of the sample.

SMS group

N -100

Control group

N -154

Age, years; mean (SD)

38.6 (10.2)

40.6 (11.5)

Gender, male; N (%)

65 (65.0)

104 (67.5)

Residential situation, living alone; N (%)

13 (13.0)

27 (17.5)

Marital status, single; N (%)

76 (76.0)

112 (72.7)

Educational level Never attended school

2 (2.0)

14 (9.1)

Primary

54 (54.0)

87 (56.5)

Secondary

29 (29.0)

37 (24.0)

University

15 (15.0)

16 (10.1)

Duration of illness, years; mean (SD)

12.1 (9.0)

13.7 (9.8)

Number of previous hospitalizations;

1.7 (2.1)

1.4 (1.9)

mean (SD)

Schizophrenia subtype; N (%) Paranoid

80 (80.0)

124 (80.5)

Undifferentiated

15 (15.0)

15 (9.7)

Disorganized

0 (0.0)

3 (2.0)

Residual

4 (4.0)

11 (7.1)

Catatonic

1 (1.0)

1 (0.7)

Treatment with atypical antipsychotic;

95 (95.0)

152 (98.7)

N (%)

Table 2

Baseline measures in adherence, attitude towards medication, insight, clinical severity and quality of life of the sample.

Variables mean (95% CI)

SMS group N- 100

Control group N- 154

MAQ total score

2.2 (2.02, 2.38)

2.2 (2.06, 2.34)

DAI-10 score

3.4(2.49, 4.31)

3.1 (2.43, 3.77)

SUMD general items score

6.2 (5.53, 6.87)

6.9 (6.42, 7.38)

CGI-SCH-SI score

Positive symptoms

2.5 (2.24, 2.76)

2.8 (2.61, 2.99)

Negative symptoms

3.3 (3.06, 3.54)

3.4 (3.22, 3.58)

Depressive symptoms

2.3 (2.06, 2.54)

2.3 (2.11, 2.49)

Cognitive symptoms

2.8 (2.58, 3.02)

3.0 (2.84, 3.16)

Global symptoms

3.0 (2.78, 3.22)

3.2 (3.06, 3.34)

EQ-5D VAS

65.9 (62.5, 69.2)

64.3 (61.7, 66.8)

MAQMorisky Green Adherence Questionnaire, DAI-10Drug Attitude Inventory 10-item, SUMD-Scale to Assess Unawareness of Mental Disorder, CGI-SCH-SI-Clinical Global Impression - Schizophrenia - Severity, EQ-5D VASEuroqol 5D, visual analog scale.

4. Discussion

This study shows that daily treatment reminders via SMS are efficacious in the enhancement of adherence to oral antipsycothic treatment in a sample of stable outpatients with schizophrenia and suboptimal adherence.

Improvement in adherence among patients assigned to the intervention remained significantly greater than routine clinical care after the 6 months of follow-up. Nevertheless, differences in adherence 3 months after stopping the intervention were smaller between the two groups. In addition, the intervention did not remain in the final linear regression model, indicating that the effect of sending SMS reminders seemed to attenuate 3 months after stopping them.

According to behavioural learning theory, sending daily SMS reminders is an intervention aimed at increasing medication adherence by modifying patient behaviour through the reception of external stimuli. Once the behaviour of taking medication is maintained, the habit will be acquired. Accordingly, this intervention should be useful for patients who are non-adherent mainly because they forget to take their medication owing to a lack of acquired routines or cognitive deficits. This mechanism could explain the improvement in those patients who changed to a negative response in MAQ item 1 (‘‘Do you ever forget to take your medicine?”). However, this mechanism does not explain the improvement experienced in those patients who were reluctant to take the medication for other reasons at baseline and changed to a negative response in MAQ item 4 (‘‘Sometimes if you feel worse when you take the medicine, do you stop taking it?”) after the intervention.

There was significant improvement in attitude towards the medication among patients who received SMS reminders. This intervention was very likely useful in increasing adherence among patients who simply forget to take their medication, and was unhelpful for those patients who refuse to take it because they are not aware of the benefits, as previously explained. Thus, the improvement in attitude towards medication observed in this study cannot be explained as a direct effect of the intervention, but as an indirect consequence. The self-observed clinical improvement and secondary improvement in quality of life achieved by regularly

Table 3

Mean changes from baseline to months 3 and 6 in primary and secondary endpoints.

Variables mean (95% CI)

Month 3

Month 6

SMS group

Control

group

p value

SMS group

Control group

p value

MAQ total score

-1.0 (-1.02, - 0.98)

- 0.7 (-0.72, - 0.68)

0.02

-1.1 (- 1.12, - 1.08)

- 0.8 (- 0.81, - 0.78)

0.04

DAI-10 score

2.0 (1.94, 2.06)

0.4 (0.35, 0.45)

0.0003

2.3 (2.24, 2.36)

0.9 (0.85, 0.95)

0.002

SUMD score

-0.8 (-0.82, -0.78)

-0.8 (-0.82, -0.78)

0.97

- 0.9 (- 0.92, -0.88)

-1.0 (-1.02, - 0.98)

0.18

CGI-SCH-SI

Positive

-0.4 (- 0.42, - 0.38)

-0.3 (- 0.32, - 0.28)

0.26

- 0.3 (- 0.32, -0.28)

- 0.3 (- 0.32, - 0.28)

0.89

Negative

- 0.4 (- 0.42, - 0.38)

- 0.3 (- 0.32, - 0.28)

0.16

- 0.6 (- 0.62, - 0.58)

-0.3 (- 0.32, -0.28)

0.03

Depressive

- 0.2 (0.22, - 0.18)

-0.1 (-0.12, -0.08)

0.09

-0.2 (-0.22, -0.18)

- 0.1 (- 0.11, - 0.08)

0.35

Cognitive

- 0.4 (- 0.42, - 0.38)

- 0.3 (- 0.32, - 0.28)

0.13

- 0.4 (- 0.42, -0.38)

- 0.3 (-0.32, - 0.28)

0.48

Global

-0.5 (- 0.52, - 0.48)

-0.3 (- 0.32, - 0.28)

0.11

- 0.5 (- 0.52, -0.48)

- 0.4 (- 0.42, - 0.38)

0.48

CGI-SCH-DC

Positive

3.2 (3.0, 3.40)

3.4 (3.24, 3.56)

0.1

3.4 (3.38, 3.42)

3.3 (3.14, 3.46)

0.63

Negative

3.3 (3.10, 3.50)

3.5 (3.36, 3.64)

0.02

3.4 (3.38, 3.42)

3.4 (3.24, 3.56)

0.82

Depressive

3.3 (3.10, 3.50)

3.5 (3.34, 3.66)

0.07

3.4 (3.38, 3.42)

3.4 (3.22, 3.58)

0.88

Cognitive

3.3 (3.12, 3.48)

3.6 (3.46, 3.74)

0.01

3.5 (3.48, 3.52)

3.5 (3.34, 3.66)

0.8

Global

3.2 (3.02, 3.38)

3.5 (3.36, 3.64)

0.012

3.3 (3.10, 3.50)

3.5 (3.32, 3.68)

0.32

EQ-5D VAS

6.6 (6.3, 6.8)

3.1 (2.91, 3.29)

0.03

6.1 (5.84, 5.36)

5.6 (5.42, 5.78)

0.75

MAQMorisky Green Adherence Questionnaire, DAI-10DrugAttitude Inventory 10-item, SUMDScale to Assess Unawareness of Mental Disorder, CGI-SCH-SI-Clinical Global Impression - Schizophrenia - Severity, CGI-SCH-DC-Clinical Global Impression - Schizophrenia - Degree of Change, EQ-5D VASEuroqol 5D, visual analog scale.

Table 4

Morisky Green items: proportion of affirmative responses at baseline, months 3 and 6.

MAQ items


Baseline


Month 3


Month 6


SMS


Control


SMS


Control


SMS Control


(N100)    (N— 154)       (N100)    (N154)       (N99)     (N— 152)

Affirmative response26, N (%)

taking it?


47 (47.0%)

13 (13.0%)

15 (15.0%)

45 (45.0%)


87 (56.5%)

30 (19.5%)

27 (17.5%)

91 (59.1%)


42 (42.4%)

12 (12.1%)

17 (17.2%)

39 (39.4%)


77 (50.6%)

29 (19.1%)

31 (20.4%)

77 (50.6%)


Table 5

Results of the step-wise multiple linear regression model at months 3 and 6.

Variables

Coefficient

95% CI

p value

Month 3

Number of hospitalizations

- 0.1

- 0.2, -0.1

0.001

SMS Group

0.3

0.1, 0.5

0.01

Baseline MAQ total score

- 0.6

- 0.8, 0.5

< 0.001

Previous hospitalization

- 0.4

- 0.7, -0.5

0.02

Baseline DAI-10 score

0.1

0.0, 0.1

< 0.001

Gender (female)

- 0.3

- 0.5, -0.1

0.02

Month 6

Previous hospitalization

- 0.3

- 0.6, -0.0

0.03

Baseline DAI-10 score

0.1

0.0, 0.1

< 0.001

Baseline MAQ total score

- 0.7

- 0.8, - 0.5

< 0.001

Gender (female)

-0.3

- 0.6, - 0.0

0.02

Occupational level

-0.3

-0.5, - 0.0

0.04

MAQMorisky Green Adherence Questionnaire, DAI-10Drug Attitude Inventory 10-item.

taking medication may be hypothesised to be influential in securing better awareness of the benefits of the medication. Indeed that was the case, and the 3-month intervention of sending SMS reminders led to improvements in several clinical indicators as well, including psychopathology and quality of life. These improvements were attenuated after stopping the intervention, in a way similar to improvement in adherence.

By contrast, an increase in awareness of illness could not be observed at either months 3 or 6. Poor insight is a complex and multidimensional phenomenon including recognition of symptoms, recognition of mental illness and even recognition of the need for treatment (Lysaker et al., 2009). The scale used in this study to measure awareness of illness (SUMD) is in accordance with this multidimensional approach of insight and evaluates the need for medication as well. Nevertheless, as previously shown, patients in our study did improve attitude towards medication. Thus, the lack of improvement in SUMD in those patients who received SMS reminders very likely reflects a lack of effect on acknowledgement of illness itself. This apparent paradox has been previously described, and it has been suggested that awareness of illness may be independent of and does not imply the acceptance of the risk and benefits of medication (Sajatovic et al., 2002; Beck et al., 2011), and may very likely apply the other way around, as well: patients may feel better with medication as a consequence of the reduction of symptoms, but this does not necessarily imply an acknowledgement of illness.

Some known factors consistently associated with a lack of adherence were found to be associated with a lesser improvement with the intervention: larger numbers of hospitalisations, previous need for hospitalisation or worse previous adherence (Lacro et al., 2002). On the other hand, some factors not usually related to non-adherence, such as the female gender or having higher-level jobs, were associated with lesser improvement in adherence at 3 and even 6 months. It can be hypothesised that patients with better-preserved functionality, as is described for women with respect to men with schizophrenia (Tamminga et al., 1997), and being able to support their own routines, such as working, would obtain lesser benefits from an intervention based on receiving a reminder of the medication at a fixed time of day to acquire a routine.

Our study has several limitations. First, the sample population comprised clinically stable patients taking a single oral antipsychotic drug. The results may thus not be generalised to less stable subjects or to those receiving several antipsychotic drugs. Unstable patients, the presence of severe paranoid delusions and/or hallucinations could affect willingness to participate or even worsen clinical status due to periodic SMS reminders. Second, we did not include prior experiences with antipsychotic treatments and side effects as factors that might be associated with therapeutic adherence. Third, although possible bias of the sample population towards a more adherent subgroup of patients was partially avoided by recruiting patients with suboptimal adherence at baseline, it is very likely that the most nonadherent patients were reluctant to participate in the study. Fourth, we measured treatment adherence using a self-rated scale. Several methods for measuring adherence are available, each with its own set of limitations (Kikkert et al., 2008). Although Jonsdottir et al. (2010) described concordance between subjective and objective assessments of adherence, it is conceivable that we may have overestimated the real adherence. In addition, adherence improvement could be explained by other factors such as social desirability of the patients or rater expectations. Currently, it has been recommended that a mixed method, using both objective and subjective methods for assessing adherence may be the most reliable option (Kikkert et al., 2008Velligan et al., 2009). Fifth, the non-blinded design used could also contribute to bias the results towards greater improvement in adherence rate and psychopathology characteristics of those patients assigned to the intervention group. Sixth, patients who take the medication early in the morning or late in the evening could not benefit from our fixed twice-a-day schedule of SMS prompting.

The 66 patients classified as ‘not properly exposed' to the intervention did not show relevant baseline socio-demographic or clinical differences compared to the final intention-to-treat population, except for a slightly worse therapeutic adherence and attitude towards medication, as well as a greater lack of insight (mean MAQscore2.5, 95%CI 2.26, 2.74; mean DAI-10 score0.9, 95%CI 0.0, 2.0; and mean SUMD score7.7, 95%CI 6.9, 9.4, respectively). Multiple causes were involved, such as the mobile phone being turned off, not working or broken or having been sold by the patient. Despite not having properly received the intervention, those excluded patients experienced adherence improvement at month 3 with a mean MAQ score of 2.4, and 17 of them (25.7%) achieved a mean MAQ score < 1.

Despite these limitations, the study also has several strengths. The sample size was large enough to allow adequate statistical power to clearly determine the efficacy of the intervention. Furthermore, the study sample showed typical characteristics observed in prevalence studies of treated populations with schizophrenia: mean age in the early forties, a slight male predominance and relatively few married people. The use of randomisation made it possible to obtain two groups of patients with very similar baseline characteristics. Furthermore, secondary outcome indicators such as quality of life and attitude towards medication were also examined to gain a better understanding of the effect of the intervention.

5. Conclusions

An SMS-based strategy offered a significant advantage for improving adherence to antipsychotic treatment over usual care. It was feasible and easy to implement. This kind of intervention could be a complimentary step-up strategy for optimising therapeutic adherence in patients with schizophrenia.

Role of funding source

This study was funded by AstraZeneca Spain.

Contributors

Author Maurino developed the research question, designed the study and wrote the protocol. Author Medina performed the statistical analyses. Authors Montes and Maurino wrote the first draft of the manuscript. All authors contributed to and have approved the final manuscript.

Conflict of interest

E Medina and J Maurino are employees of AstraZeneca Spain. JM Montes has received grants and served as consultant, advisor or CME speaker for the following entities: AstraZeneca, Boehringer Ingelheim, Bristol-Myers-Squibb, Otsuka, Lundbeck, Pfizer and Robi y Servier. M Gomez-Beneyto does not have any conflict of interest to declare.

Acknowledgements

The authors thank Ruben Ampudia for his support in the statistical analysis.

References

Amador, X.F., Flaum, M., Andreasen, N.C., Strauss, D.H., Yale, S.A., Clark, S.C., Gorman, J.M., 1994. Awareness of illness in schizophrenia and schizoaffective and mood disorders. Archives of General Psychiatry 51, 826-836.

Ascher-Svanum, H., Faries, D.E., Zhu, B., Ernst, F.R., Swartz, M.S., Swanson, J.W., 2006. Medication adherence and long-term functional outcomes in the treatment of schizophrenia in usual care. Journal of Clinical Psychiatry 67, 453-460.

Badia, X., Roset, M., Montserrat, S., Herdman, M., Segura, A., 1999. The Spanish version ofEuroQoL: a description and its applications. European Quality ofLife Scale. Medicina Clinica (Barc) 112 (suppl. 1), 79-85.

Barkhof, E., Meijer, C.J., de Sonneville, L.M.J., Linszen, D.H., de Hann, L., 2012. Interventions to improve adherence to antipsychotic medication in patients with schizophrenia-A review of the past decade. European Psychiatry 27, 9-18.

Borzekowski, D.L.G., Leith, J., Medoff, D.R., Potts, W., Dixon, L.B., Balis, T., Hackman, A.L., Himelhoch, S., 2009. Use of the internet and other media for health information among clinic outpatients with serious mental illness. Psychiatric Services 60, 1265-1268.

Christensen, H., Hickie, I.B., 2010. E-mental health: a new era in delivery of mental health services. Medical Journal of Australia 192 (11 suppl), S2-S3.

Cramer, J.A., Rosenheck, R., 1998. Compliance with medication regimens for mental and physical disorders. Psychiatric Services 49, 196-201.

Depp, C.A., Mausbach, B., Granholm, E., Cardenas, V., Ben-Zeev, D., Patterson, T.L., Lebowitz, B.D., Jeste, D.V., 2010. Mobile interventions for severe mental illness: design and preliminary data from three approaches. Journal of Nervous and Mental Disease 198, 715-721.

Dolder, C.R., Lacro, J.P., Leckband, S., Jeste, D.V., 2003. Interventions to improve antipsychotic medication adherence: review of recent literature. Journal of Clinical Psychopharmacology 23, 389-399.

Ehrenreich, B., Righter, B., Rocke, D.A., Dixon, L., Himelhoch, S., 2011. Are mobile phones and handheld computers being used to enhance delivery of psychiatric treatment? Journal of Nervous and Mental Disease 199, 886-891.

Erickson, S.R., Coombs, J.H., Kirking, D.M., Azimi, A.R., 2001. Compliance from selfreported versus pharmacy claims data with metered-dose inhalers. Annals of Pharmacotherapy 35, 997-1003.

Goff, D.C., Hill, M., Freudenreich, O., 2010. Strategies for improving treatment adherence in schizophrenia and schizoaffective disorder. Journal of Clinical Psychiatry 71 (suppl 2), 20-26.

Gray, R., Leese, M., Bindman, J., Becker, T., Burti, T., David, A., Gournay, K., Kikkert, M., Koeter, M., Puschner, B., Schene, A., Thornicroft, G., Tansella., M., 2006. Adherence therapy for people with schizophrenia. British Journal of Psychiatry 189, 508-514.

Haro, J.M., Kamath., S.A., Ochoa, S., Novick, D., Rele, K., Fargas, A., et al., 2003. The Clinical Global Impression-Schizophrenia Scale: a simple instrument to measure the diversity of symptoms present in schizophrenia. Acta Psychiatrica Scandinavica 107 (suppl. 416), 16-23.

Harrison, V., Proudfoot, J., Wee, P.P., Parker, G., Pavlovic, D.H., Manicavasagar, V., 2011. Mobile mental health: Review of the emerging field and proof of concept study. Journal of Mental Health. [Epub ahead of print].

Hogan, T.P., Awad, A.G., 1992. Subjective response to neuroleptics and outcome in schizophrenia: a re-examination comparing two measures. Psychological Medicine 22, 347-352.

Humphrey Beebe, L., Smith, K., Bennett, C., Bentley, K., Walters, A.B., Hancock, B., Farmer, S.V., Earle, K., White, S., 2010. Cell phone use in people with schizophrenia spectrum disorders. Journal of Psychosocial Nursing 48 (4), 32-37.

Jonsdottir, H., Opjordsmoen, S., Birkenaes, A.B., Engh, J.A., Ringen, P.A., Vaskinn, A., Aamo, T.O., Friis, S., Andreassen, O.A., 2010. Medication adherence in outpatients with severe mental disorders: relation between self-reports and serum level. Journal of Clinical Psychopharmacology 30 (2), 169-175.

Julius, R.J., Novitsky, M.A., Dubin, W.R., 2009. Medication adherence: a review of the literature and implications for clinical practice. Journal of Psychiatric Practice 15, 34-44.

Kikkert, M.J., Barbui, C., Koeter, M.W.J., 2008. Assessment of medication adherence in patients with schizophrenia. The Achilles heel of adherence research. Journal of Nervous and Mental Disease 196, 274-281.

Konig, H.H., Roick, C., Angermeyer, M.C., 2007. Validity of the EQ-5D in assessing and valuing health status in patients with schizophrenic, schizotypal or delusional disorders. European Psychiatry 22, 177-187.

Lacro, J.P., Dunn, L.B., Dolder, C.R., Leckband, S.G., Jeste, D.V., 2002. Prevalence of and risk factors for medication non-adherence in patients with schizophrenia: a comprehensive review of recent literature. Journal of Clinical Psychiatry 63, 892-909.

Leucht, S., Heres, S., 2006. Epidemiology, clinical consequences, and psychosocial treatment of non-adherence in schizophrenia. Journal of Clinical Psychiatry 67, 3-8.

Liu-Seifert, H., Osuntokun, O.O., Godfrey, J.L., Feldman, P.D., 2010. Patient perspectives on antipsychotic treatments and their association with clinical outcomes. Patient Preference and Adherence 4, 369-377.

Lysaker, P.H., Buck, K.D., Salvatore, G., Popolo, R., Dimaggio, G., 2009. Lack of awareness of illness in schizophrenia: conceptualizations, correlates and treatment approaches. Expert Review of Neurotherapeutics 9, 1035-1043.

Makela, K., Paavola, T., Stenman, M., 2010. Development of short message service application for patient-provider communication in clinical psychiatry. Telemedicine J E Health 16 7, 827-829.

Montes, J.M., Maurino, J., Diez, T., Saiz-Ruiz, J., 2011. Factors associated with the effectiveness of a telephone-based nursing strategy for enhancing medication adherence in schizophrenia. Clinical Practice and Epidemiology in Mental Health 7, 117-119.

Morisky, D.E., Green, L.W., Levine, D.M., 1986. Concurrent and predictive validity of a self-reported measure of medication adherence. Medical Care 24, 67-74.

Patrick, K., Griswold, W.G., Raab, F., Intille, S.S., 2008. Health and the Mobile Phone. American Journal of Preventive Medicine 35, 177-181.

Patterson, T.L., Leeuwenkamp, O.R., 2008. Adjunctive psychosocial therapies for the treatment of schizophrenia. Schizophrenia Research 100, 108-119.

Pijnenborg, G.H., Withaar, F.K., Timmerman, M.E., Van den Bosch, R.J., Evans, J.J., 2010. The efficacy of SMS text messages to compensate for the effects of cognitive impairments in schizophrenia. British Journal of Clinical Psychology 49 (Pt 2), 259-274.

Ruiz, A.J., Pousa, E., Duno, R., Crosas,J.M., Cuppa, S., Garcia-Ribera, C., 2008. Spanish adaptation of the Scale of Unawareness of Mental Disorder (SUMD). Actas Espanolas de Psiquiatria 3, 111-119.

Robles Garcia, R., Salazar Alvarado, V., Paez Agraz, F., Ramirez Barreto, F., 2004. Assessment of drug attitudes in patients with schizophrenia: psychometric properties of the DAI Spanish version. Actas Espanolas de Psiquiatria 32, 138-142.

Sajatovic, M., Rosch, D.S., Sivec, H.J., Sultana, D., Smith, D.A., Alamir, S., Buckley, P., Bingham, C.R., 2002. Insight into illness and attitudes toward medications among inpatients with schizophrenia. Psychiatric Services 53, 1319-1321.

Smith, J.C., Schatz, B.R., 2010. Feasibility of mobile phone-based management of chronic illness. AMIA Annual Symposium Proceedings, 757-761.

Spaniel, F., Vohlidka, P., Hrdlika, J., Kozeny, J., Novak, T., Motlova, L., Cermak, J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: Information technology aided release prevention programme in schizophrenia. Schizophrenia Research 98, 312-317.

Tamminga, C.A., 1997. Gender and schizophrenia. Journal of Clinical Psychiatry 58 (suppl 15), 33-37.

Van den Berg, N., Graba, H.J., Freyberger, H.J., Hoffmann, W., 2011. A telephone-and text-message based telemedical care concept for patients with mental health disorders- study protocol for a randomized, controlled study design. BMC Psychiatry 11, 30.

Velligan, D.I., Weiden, P.J., Sajatovic, M., Scott, J., Carpenter, D., Ross, R., Docherty, J.P., 2009. The expert consensus guideline series: Adherence problems in patients with serious and persistent mental illness. Journal of Clinical Psychiatry 70 (suppl 4), 1-46.

Van Gent, E., Knoppert Van Der Klein, E., 2010. Improving compliance and treatment outcome in bipolar and schizophrenic patients by using Short Message Text Service (SMS). International Journal of Psychiatry in Clinical Practice 14 (suppl 1), 39.

Wangberg, S.C., Bergmo, T.S., Johnsen, J.A., 2008. Adherence in internet-based interventions. Patient Preference and Adherence 2, 57-65.

SCHRES-06819; No of Pages 7


Schizophrenia Research xxx (2016) xxx-xxx



Contents lists available at ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres

Effects of online intervention for depression on mood and positive symptoms in schizophrenia^

Steffen Moritz a*1, Johanna Schroder a-1, Jan Philipp Klein b, Tania M. Lincolnc, Christina Andreou a,

Anja Fischer d'27 28 29, Sonke Ari' a'29

a University Medical Center Hamburg-Eppendorf, Department of Psychiatry and Psychotherapy, Martinistrasse 52, D-20246 Hamburg, Germany

b Department of Psychiatry and Psychotherapy, Lubeck University, Ratzeburger Allee 160,23538 Lubeck, Germany

c Clinical Psychology and Psychotherapy, Institute of Psychology, University of Hamburg, 20146 Hamburg, Germany

d King's College London, Institute of Psychiatry, Psychology, and Neuroscience, Department of Health Psychology, Guy's Hospital, St Thomas Street, London SE1 9RT, United Kingdom

ARTICLE INFO


ABSTRACT


Article history:

Received 17 September 2015

Received in revised form 12 April 2016

Accepted 19April2016

Available online xxxx


Background: Depression is common in schizophrenia. Whereas the improvement of mood and self-esteem represents a subjective treatment priority for many patients, depression is rarely a primary target for clinical intervention. The present trial examined whether an online intervention for depression can ameliorate depressive symptoms in schizophrenia.

Keywords: Schizophrenia Depression

Online intervention

Self-help

Psychosis


Methods: A total of 58 individuals with schizophrenia were invited to participate in an online survey which encompassed the Center for Epidemiologic Studies-Depression Scale (CES-D, primary outcome), the PatientHealth-Questionnaire-9 (PHQ-9) and the Paranoia Checklist. Subsequently, telephone interviews were conducted to verify diagnostic status and assess symptoms (Positive and Negative Syndrome Scale, PANSS). Participants were randomized either to the experimental condition (online depression intervention) or to a waitlist control condition. Three months after inclusion, a reassessment was carried out (self-report and telephone interview blind for group condition). The trial was registered (registration: DRKS00007888).

Results: Participants in the treatment group showed a significant decline of depressive symptoms at a medium-to-large effect size, as assessed with the CES-D and the PANSS depression item, in comparison to the waitlist control group (completer (CC) and intention-to-treat analyses (ITT)). For the PHQ-9 (CC and ITT) and the PANSS distress subscale (CC only) significance was bordered at a medium effect size. Completion at the post-assessment after three months was 84%.

Discussion: Depression in schizophrenia is both underdiagnosed and undertreated. To reduce the large treatment gap in the disorder, low threshold strategies are urgently needed. Online treatment and bibliotherapy may represent valuable tools to address patients' needs beyond the treatment of the core positive syndrome.

© 2016 Published by Elsevier B.V.

1. Introduction

1.1. Depression in schizophrenia

Delusions and hallucinations (i.e., positive symptoms) are the defining features of psychosis and until recently represented the conventional target for the treatment of schizophrenia (Suzuki, 2011). In contrast, depressive symptoms in this disorder received comparatively little attention, presumably owing to historical and diagnostic reasons. Since Kraepelin (1899) who separated schizophrenia (then called dementia praecox) from affective disorders and more recently Schneider (1959) who confined the core of schizophrenia to positive features, depression is often regarded a negligible and secondary symptomatic feature of psychosis. Moreover, some phenomena that would be counted as depressive symptoms in other disorders are labeled as negative symptoms in schizophrenia (e.g., social withdrawal) or even receive distinct diagnostic terms once a diagnosis of schizophrenia has been determined (Burckhardt, 2012; Kuck et al., 1992). For example, a mental state dominated by affective numbness is usually termed melancholia in depression but anhedonia in psychosis. Similarly, lack of drive (depression) is commonly relabeled as avolition in schizophrenia and again counted as a negative symptom. This, along with the longstanding preoccupation that psychosis is not amenable to understanding (Jaspers, 1973; Walker, 1991), may have contributed to the relative neglect of treatment of depressive symptoms in psychosis in the past and to the poor transfer of available psychological treatment options against depression in patients with psychosis.

http://dx.doi.org/10.1016/j.schres.2016.04.033

0920-9964/© 2016 Published by Elsevier B.V.

S. Moritz et al. / Schizophrenia Research xxx (2016) xxx-xxx

1.2. Reasons why depression is an important treatment target in schizophrenia

A number of reasons exist why depressive symptoms deserve greater attention in the treatment of psychosis. First, beginning with Kasanin (1933), who coined the term schizoaffective disorder, clinicians increasingly acknowledge that affective symptoms coexist with psychosis. A review estimated that at least 50% of patients suffer from comorbid depression (Buckley et al., 2009), and many patients show single depressive symptoms such as low self-esteem (Freeman et al., 1998; Kesting and Lincoln, 2013; Moritz et al., 2010). Second, suicidality, a grave and life-threatening manifestation of depression, is frequent in psychosis and approximately 5% of patients commit suicide (Hor and Taylor, 2010). While some suicides are due to the influence of acoustic hallucinations, depression represents the best predictor for suicidality or selfharm behaviors in patients with schizophrenia (Fusar-Poli et al., 2014). Third, the implicit prevailing treatment paradigm posits that improving positive symptoms and insight will raise quality of life and reduce depression. Research suggests, however, that enhancement of insight can even paradoxically aggravate affective problems (Karow et al., 2008; Lincoln et al., 2007). Moreover, depression is not just a secondary consequence of having a severe mental disorder, it is also a frequent premorbid precursor of psychosis (Fusar-Poli et al., 2014). Fourth, depressive symptoms are often formulated by patients as their preferred target of treatment (Byrne et al., 2010; Hafner et al., 2013; Sterk et al., 2013; Moritz et al., in press-b).

1.3. Poor efficacy of available treatment

Notwithstanding the need to address depression in the therapy of psychosis as well as guidelines fostering psychotherapy in psychosis, many patients are still deprived of psychotherapy in general (Bechdolf and Klingberg, 2014; Shafran et al., 2009). If psychotherapy is sought at all, it often targets positive symptoms. Both pharmacological and psychological treatment approaches for psychosis are only partially effective in reducing depression. Antipsychotics yield a small effect size for the improvement of depressive symptoms in schizophrenia (Leucht et al., 2009). In fact, antipsychotic medication may even induce pharmacological depression (Naber and Karow, 2001). Augmentation with antidepressants has been shown ineffective for the treatment of depressive symptoms according to two meta-analyses (Kishi et al., 2013; Kishi and Iwata, 2014), underlining the need for non-pharmacological treatment options for depression in psychosis patients. Cognitive-behavioral therapy (CBT) yields a small to medium effect size (0.36) for the improvement of mood (Wykes et al., 2008).

1.4. The present study

The present study explored whether a generic online intervention for depression administered in a non-clinical setting can reduce depressive symptoms in patients with psychosis. We used a program called HelpID (developed by the novego AG) which is based on the CBT theoretical framework. Meta-analyses show that online interventions for depression exert a small-to-medium effect size in patients with depression when administered unguided (Cuijpers etal., 2011; Richards and Richardson, 2012, Johansson and Andersson, 2012).

Whether these effects also hold true for patients with comorbid depressive symptoms alongside other primary disorders, like obsessive-compulsive disorder, borderline personality disorder and schizophrenia, awaits to be established. While we hypothesize that HelpID will reduce depressive symptoms in psychosis, we were unable to make predictions with respect to the magnitude of the effect. Although self-help and online interventions have proven feasible in psychosis (Alvarez-Jimenez et al., 2014), a limiting factor could be that the online intervention under investigation is not adapted to the specific problems (e.g., stigma and self-stigma) and deficits (e.g., cognitive dysfunctions which may compromise translation of lessons/learning aims into everyday life) of psychotic patients. However, as patients with psychosis often do not receive specific treatment for depression, the potential of the program may be higher than in conventional (depression) populations who are usually not naive about the contents of such programs. In line with this, the magnitude of the effect of online interventions for depression was moderate for a group of neurological patients in two recent studies of whom most had never received specific treatment for depression before (Fischer et al., 2015; Schroder et al., 2014). Based on the assumption that depressive symptoms and depression-related cognitions (e.g., worry thinking style, negative beliefs about the self, interpersonal sensitivity, sleep disturbance) play a causal role for the emergence of positive symptoms (Garety et al., 2001; Freeman and Garety, 2014; Lincoln et al., 2014), we also expected the treatment to impact on positive symptoms.

2. Method

2.1. Participants

Participants were primarily recruited via a database of former patients from the Department of Psychiatry and Psychotherapy at the University Medical Center Hamburg-Eppendorf (Germany, UKE). All former patients had previously given written informed consent to be re-contacted for future studies. While for most individuals a diagnosis of schizophrenia had been established on prior occasions, we additionally administered a diagnostic telephone interview as a precondition for participation. We also contacted high-quality Internet forums (e.g. moderated, conveying evidence-based information) devoted to schizophrenia as an additional source to recruit study participants. The following inclusion criteria applied: age between 18 and 65 years, willingness to participate in two anonymous (Internet-based) surveys as well as diagnostic telephone interviews that were scheduled three months apart, and a diagnosis of schizophrenia (as verified by telephone interview). Moreover, participants had to experience present subjective depressive symptoms and a willingness to undergo treatment for these symptoms (no formal cut-off was set). While presence of depressive symptoms was an explicit inclusion criterion, a diagnosis of major depression or dysthymic disorder was not. Severe suicidality led to exclusion. In these cases, participants were informed about help lines and treatment options. The trial was set up as an add-on intervention to care-as-usual: Participants were informed that the trial would not interfere with current treatments. For example, individuals were allowed to continue to take medication or see a physician.

Interested individuals were directed to the baseline survey via a weblink. The survey was set up using questback®, a software allowing to create online surveys. The study was anonymous (no name or postal address was requested; we did no store IP addresses were stored). Participants were informed that they would either immediately receive an online code allowing a free 3-month access to HelpID or would be allocated to a waitlist control condition. The latter group was promised full access to the program subsequent to the post-assessment (online survey and interview). Group allocation was carried out in random fashion subsequent to baseline assessment (i.e. following the interview) using an automated randomization plan with no stratification.

The trial was registered at the Internet Portal of the German Clinical Trials Register (DRKS; DRKS00007888). The DRKS was approved for the primary register in the WHO network and thus meets the requirements of the International Committee of Medical Journal Editors (ICMJE).

2.2. Procedure

On the first page of the baseline survey, the rationale of the study as well as exclusion criteria were summarized. All participants provided electronic informed consent. Multiple log-ins via the same computer were prevented by means of cookies. The survey consisted of the

S. Moritz et al. / Schizophrenia Research xxx (2016) xxx-xxx

following parts: demographic section (e.g., gender, age), medical history (e.g., psychiatric diagnoses), assessment of psychopathology (see questionnaires section below), request for an email address (to match baseline and post-survey data), telephone number and contact information (e.g., preferred times for the interview) and whether prior responses had been correct. Following completion of the baseline assessment, interviewers tried to reach individuals at their preferred times to carry out the telephone interview (the assessments are described below). Telephone interviews were conducted blinded for group allocation and patients were explicitly reminded at the reassessment not to disclose which condition they had been allocated to.

Following the blinded interview (see below) eligible participants were randomized (see above). Participants were emailed by a researcher (JS) not involved in the interviews and informed about whether they were allocated to the intervention group (a voucher providing access to the program was included along with further instructions) or the waitlist control group.

Three months after the baseline interview, participants were contacted via email for participation in the post-survey. Up to two reminders were dispatched if participants failed to complete the postassessment.

For the post-survey, individuals were requested to first enter their email address to allow matching of baseline and post data. The postassessment consisted of the following parts: introduction, questionnaire on psychopathology (see below) and evaluation of the online intervention (if participants stated that they had logged in to the system at least once). At the end, participants were asked whether they had made truthful responses and were given the opportunity to leave comments (e.g., if they experienced technical problems). They were then asked for their telephone number again and for their preferred times for the final interview. Following the final blinded interview, participants in the waitlist control group received an email containing a code giving full access to the program. All participants received a relaxation manual with audio files as an additional incentive.

2.3. HelpID

The Internet intervention Help ID (developed by novego AG, Seevetal, Germany) involves 12 weekly scheduled modules, which are derived from a pool of 17 modules (see below; 7 of these modules are mandatory). The individual set is composed according to the replies from a pre-assessment containing 60 questions which is performed prior to starting the program. Thus, every user receives a program tailored to their individual needs. Each module requires 45 to 60 min to complete and includes 14-19 pages of text. A video moderation leads through the program, which also includes interactive exercises and practice sheets, illustrations, photographs, animations and audios. For standard use of the program, motivational SMS (optional), email reminders and personal feedback on individual questions in a protected area are provided by the psychological team. For the present study however, no personal feedback was given as we wanted to evaluate the efficacy of a fully unguided treatment program.

The program is based on a combination of therapeutic methods derived from CBT, acceptance and commitment therapy (ACT) as well as systemic counseling and therapy also including relaxation audio files and music therapy, provided using an individualized algorithm considering gender-specific and symptom-related aspects, somatic variables (such as chronic back pain or cardiac arrhythmia) and potentially also postpartum depression. Additional modules on, for example, heart problems and post-partum depression were presented if corresponding cue questions (here, on diagnosed heart deficits or the birth of a child in the last year) were affirmed. The algorithm also took into account responses on the PHQ. Essential contents and goals of the program are to convey an understanding of depression by means of psychoeducation, the development of alternative viewpoints fostering activation in everyday life, strengthening social relationships as well as attention and relapse prevention exercises.

The titles of the 17 modules are as follows (modules that are underlined are mandatory and administered to every individual): the way out of depression getting started if you have the blues” • depression pleasant things in everyday life learning to reward yourself how to break thought spirals together against depression recognizing yourself relaxation against depression attention - made easy learning to let go doing myself a favor sun against murky thoughts listen to your body preventing relapse therapeutic support my heart and I. While seven of the modules were presented to each individual, five additional modules were selected according to individual responses to cue questions.

2.4. Questionnaires (online assessment)

Participants were required to complete three questionnaires. The survey proceeded only if all items had been responded to.

2.4.1. Primary outcome

The Center for Epidemiologic Studies-Depression Scale (CES-D) (Hautzingerand Brahler, 1993; Radloff, 1977) is a20 item questionnaire covering depressive symptoms. In keeping with efforts to give patients' preferences and assessment greater consideration (Karamatskos et al.,

2.4.2. Secondary outcomes

The Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001) was assessed as an additional index of depression. The PHQ-9 is a selfreport instrument derived from the Primary Care Evaluation of Mental Disorders (PRIME-MD). Its nine items tap into the nine diagnostic criteria in the DSM-IV. Its psychometric properties can be judged as good with a sensitivity of 0.80 and a specificity of 0.92 (Gilbody et al., 2007).

The Paranoia Checklist (Freeman et al., 2005) consists of 18 items tapping into subclinical as well as clinical signs of paranoid beliefs and suspiciousness. The psychometric properties are good (Freeman et al., 2005; Lincoln et al., 2010). The test-retest reliability of the online version is excellent (Moritz et al., in press-a; Moritz et al., 2014) and the scale shows good internal consistency and convergent validity (Lincoln et al., 2010). In our adaption of the scale, patients were required to rate the current symptom severity on a five-point Likert scale ranging from 1 (not at all) to 5 (extremely).

2.5. Psychopathological interview

A diagnosis of schizophrenia or schizoaffective disorder was verified via telephone using the Mini-International Neuropsychiatric Interview (M.I.N.I.; Sheehan et al., 1998). Interviewers were blind to group status (intervention or control group). The M.I.N.I. has been successfully validated against other diagnostic tools (Sheehan et al., 1998). Symptom severity was measured with the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1989), which is considered the gold standard for the psychometric assessment of schizophrenia (Suzuki, 2011). The PANSS has good psychometric properties and is sensitive to change (Kay et al., 1989; Peralta and Cuesta, 1994; Santor et al., 2007). In order to avoid repetition of questions, both ratings were synthesized into one interview and not administered successively. Ratings followed semi-structured interview protocols. Before the trial, a rater training was held using several video demonstrations. In addition, the first interview was monitored by an experienced rater (JS). The same rater administered the interview for each individual patient to avoid rating biases. For the PANSS, we adopted a five-factor algorithm suggesting


Fig. 1. CONSORT chart diagram.

the following subscales: positive symptoms, negative symptoms, disorganization, excitement, and emotional distress (van der Gaag et al., 2006). The general item 6 (depression) served as an index for clinician-rated depression. However, as negative symptoms cannot be reliably assessed over the telephone, we did not calculate the PANSS negative subscale.

3. Results

A total of 174 participants accessed the first page of the survey; 105 participants provided their electronic informed consent. Seventy-nine participants completed the entire baseline survey. Of these, 21 were excluded for the reasons provided in the CONSORT chart (Fig. 1). The main reason was that patients could not be reached via telephone (n = 11) because they either did not answer the phone (n = 6), did not leave a telephone number at all (n = 2), or left a wrong telephone number (n = 3). Thus, 58 participants fulfilling inclusion criteria were randomized to the two conditions.

3.1. Baseline characteristics & attrition

The baseline characteristics of the sample are provided in Table 1. Except for a marginal difference on age, the groups did not differ on any baseline parameter. Depressive symptoms, as assessed with the PHQ-9, were moderate to severe. The PANSS total score signaled mild/ sub-acute symptomatology for most patients. The majority of participants were on medication (HelpID: 45.5% medication only, 9% psychotherapy only, 45.5% both; waitlist control: 61.1% medication only, 0% psychotherapy only, 38.9% both, x2(1) = 2.20, p = 0.33). As shown in Table 1, most individuals fulfilled diagnostic criteria for current (comor-bid) depression. The rest either were diagnosed with depressive episodes in the past, dysthymia or did not fulfill formal criteria for depression.

After three months, 84% of the sample (n = 49, HelpID: n = 25, waitlist: n = 24) were reached for the post-assessment (self-report and expert rating with the PANSS). Treatment status or concurrent treatment (e.g., lowered or enhanced dosage) did not change substantially between groups across time (p > 0.1).

3.2. Statistical analyses

In line with recommendations in the literature, we performed an ANCOVA with the pre-post difference score as the dependent variable and the respective baseline score as the covariate. This type of analysis

Table 1

Baseline characteristics. Frequency or means and standard deviations (in brackets).

HelpID depression program (n = 31)

Waitlist (n = 27)

Statistics (t = 56)

Demographic variables

Age in years

38.19

(11.78)

43.41

(8.42)

t = 1.91, p = 0.06

Gender (male/female)

17/14

10/17

X2(1) = 1.84, p = 0.17

Years of formal school education

11.94

(1.53)

11.56

(1.45)

t = 0.97, p = 0.34

Psychopathology

Comorbid depression

61%

64%

X2(1) = 0.61, p = 0.80

PANSS total (corrected for negative symptoms)

33.39

(7.47)

35.93

(9.14)

t = 1.16, p = 0.25

CES-D

56.00

(17.26)

56.26

(16.53)

t = 0.06, p = 0.95

PHQ total

10.55

(5.66)

12.44

(6.20)

t = 1.22, p = 0.23

Paranoia checklist

36.71

(18.07)

38.41

(15.52)

t = 0.39, p = 0.70



Fig. 2. CES-D: Significant decline of depressive symptoms in the intervention condition relative to the control condition with a large effect size. Bars represent standard errors.


accounts for regression to the mean and raises power of the analyses (Borm et al., 2007; Kenward and Roger, 1997).

3.3. Primary outcome (complete cases)

For the primary outcome (CES-D) a significant difference emerged, F(1,46) = 9.84, p = 0.003, qpartial = 0.176 with a large effect size in favor of the intervention group relative to the waitlist group (see Fig. 2). Paired t-tests showed that symptoms decreased significantly in the treatment (p = 0.002) but not in the control condition (p = 0.822).

3.4. Secondary outcomes (complete cases)

For the PHQ-9 a trend in favor of the treatment group emerged at a medium effect size: ANCOVA: F(1,46) = 3.71, p = 0.06, npartial = 0.075 (see Fig. 3). Paired t-tests showed that symptom decrease bordered on significance in the treatment (p = 0.052) but not in control group (p = 0.788). As the PHQ-9 is a very short scale tapping into a heterogeneous set of symptoms, we conducted exploratory item-wise comparisons which revealed three differences in favor of the intervention group relative to the waitlist control group, particularly for self-esteem: PHQ item 6 (low self-esteem; p = 0.003, npartial = 0.175), PHQ item 3 (sleep; p = 0.052, qpartial = 0.080) and PHQitem 7 (poor attention; p = 0.064, npartial = 0.073). No group differences emerged on the Paranoia Checklist across time, F(1,46) = 0.13, p = 0.91, npartial < 0.001 (see Fig. 4).

The PANSS depression item (general item 6) score decreased more strongly in the intervention group than in the waitlist control group at a medium-to-large effect size, F(1,45) = 5.07, p = 0.029, n2artial = 0.101 (see Fig. 5). We then looked at the PANSS syndrome scales, whereby a marginally significant effect emerged for the PANSS distress subscale at a medium effect size, p = 0.054, n2artial = 0.078.For all other PANSS syndrome scores, the effects were non-significant (positive: p = 0.986, disorganization: p = 0.172, excitement: p = 0.245). Exploratory

item-wise analyses revealed a significant difference in favor of HelpID for blunted affect, which however would have not withstood correction for multiple testing (PANSS item N1), F(1,46) = 5.37, p = 0.025, npartial = 0.104.

3.5. Intention-to-treat analyses

For intention-to-treat analyses considering all randomized patients, missing outcome data were imputed from information on the psycho-pathological and three demographic (age, gender, school education) indexes by the expectation-maximization (EM) algorithm, trimmed to fall between the minimum and maximum of possible values. This produced largely similar findings for all variables: CES-Q(F(1,55) = 12.023, p = 0.001, npartial = 0.179), PANSS depression (F(1,55) = 4.356, p = 0.042, n2artial = 0.073), PHQ-9 (F(1,55) = 2.901, p = 0.094, ^partial = 0.05), PANSS positive (F(1,55) = 0.161, p = 0.690, qpartial = 0.003), PANSS disorganization (F(1,55) = 2.540, p = 0.117, npartial = 0.044), PANSS excitement (F(1,55) = 3.218, p = 0.078, q2artial = 0.055), PANSS distress (F(1,55) = 2.591, p = 0.113, npartial = 0.045), Paranoia Checklist (F(1,55) = 0.061, p = 0.806, qpartial = 0.001).

3.6. Subjective appraisal

All individuals in the HelpID condition logged in to the program at least once. Three individuals acknowledged that they did not perform the exercises. Table 2 shows that more than almost two thirds treatment group would recommend the training to a friend, would use the program again, found the quality of the program good and regarded the program as being helpful in dealing with problems. Endorsement was lower for the following domains: individual help, useful for one's

S. Moritz et al. / Schizophrenia Research xxx (2016) xxx-xxx

Table 2

Feedback of the group who received the HelpID depression program (n = 25).

Appraisal

Percentage positive versus negative appraisal

How do you assess the quality of the program? (excellent, good versus not so good, bad)

72%/28%

Did you receive the help you wanted? (yes, rather yes vs. rather no, no)

56%/44%

Did the program meet your needs? (fully applies, rather yes vs. few needs met, did not meet my needs)

56%/44%

Would you recommend the program to a friend, if she/he had needed similar help? (yes, rather yes vs. rather no, no)

64%/36%

How satisfied are you with the degree of help you received from the program (very satisfied, rather satisfied vs. somewhat dissatisfied very dissatisfied)

56%/44%

Did the program help you to deal with problems more appropriately? (helped a lot, helped somewhat vs. it did not really help, no)

68%/32%

How satisfied are you with the program overall? (very satisfied, rather satisfied vs. somewhat dissatisfied, very dissatisfied)

60%/40%

Would you use the program again? (very much, I think so vs. I do not think so, not at all)

68%/32%

needs, satisfied with the degree of the help received by the program and satisfaction overall.

3.7. Adherence

A total of 16% of the sample used the program daily or almost daily, 28% used it once each week and 12% every two weeks. 24% used it once throughout the treatment period; 12% of the participants entered the program but did not complete any module. On average, patients logged in 28.71 times (SD = 46.67; range: 2-232).

3.8. Test-retest reliability

The three-month reliability was r = 0.62 for the PANSS total score (corrected for the negative items), r = 0.79 for the CES-Q, r = 0.70 for the Paranoia Checklist and r = 0.62 for the PHQ-9.

4. Discussion

Our study examined whether a generic and unguided online selfhelp intervention for depression, leads to an improvement of depressive symptoms in individuals with schizophrenia. Treatment of depression is of high importance in view of its prevalence in schizophrenia and because regulation of emotional problems represents a high treatment priority in patients (Byrne et al., 2010; Sterk et al., 2013; Moritz et al., in press-b). We combined the advantages of online research (e.g., economic implementation, facilitated access to patients of whom many would not have participated in an institutional treatment context) with that of a randomized-controlled clinical trial (RCT; diagnostic interview, external assessment with a gold-standard instrument, PANSS). We were able to recruit 58 individuals with a valid diagnosis of schizophrenia and reached 84% of the participants after three months. The majority of participants liked the program, found it helpful, would use it again and would recommend it to others, notwithstanding that the program did not address individual problems of patients with schizophrenia. This retrospective assessment was also mirrored by the main outcome: In line with our hypothesis, we observed a significant and large decline of depressive symptoms in the HelpID group relative to the waitlist control as assessed with the CES-D. While the results pointed into a similar direction (medium effect size) for the PHQ-9, the group differences only reached trend level for this scale. An exploratory post-hoc assessment of individual PHQ-9 items revealed group differences for self-esteem (PHQitem 6). For the PANSS depression item we found a significant difference with a medium-to-large effect size in favor of HelpID. The distress subscale, which also captures anxiety, revealed a trend in favor of HelpID. No significant differences were measured for the PANSS positive syndrome or Paranoia Checklist. Thus, it seems that reducing depression does not suffice to reduce positive symptoms as assessed by this scale. However, a longer follow-up investigation is currently planned to explore whether a reduction of depressive symptom is followed by improvement on positive symptoms at a later time-point (sleeper effect) as predicted by recent theoretical models (Freeman and Garety, 2014).

Before drawing conclusions and suggesting ideas for future research, several limitations should be brought to the readers' attention. First, the sample size was rather small precluding elucidation of factors moderating outcome. Second, further follow-up studies are needed, ideally ones that include an active control condition to detect whether improvement is sustained over time. Third, unlike conventional randomized controlled trials (RCTs), it is impossible to determine the screening-to-inclusion ratio and to detect how those participating in the trial differed from those who decided against it. Thus, despite encouraging evidence for the feasibility and efficacy of the approach its effectiveness needs to be replicated in a routine setting. Finally, while assessment with the PANSS is feasible for positive and depressive symptoms via the telephone, negative symptoms are hard to verify without face-to-face contact so that we could not compute scores for the PANSS negative subscale.

To conclude, the study showed that an online intervention can improve depressive symptoms in individuals with psychosis. The magnitude of the effect was larger than was expected from prior studies in patients with primary depression (Spek et al., 2007). We speculate that this is due to the naivety of the recruited individuals concerning the contents of depression treatment which permitted its full potential to unfold. In contrast, many depressed patients undergoing online interventions for depression are usually familiar with standard face-to-face therapy thereby limiting surplus effects. Of note, the effect sizes were larger than those found for conventional treatment options for depression in psychosis such as antipsychotic (small effect) and antidepressant agents (negligible effect) and CBT (small-to-medium effect). We think that the larger dosageof the program (e.g., weekly sessions over 12 weeks) might have augmented the effect. Future studies should investigate whether guided interventions as well as incorporating elements addressing the needs and special problems of schizophrenia patients (e.g., stigma, social withdrawal, distrust, voice-hearing) can enhance the observed effects and generalize to psychotic symptoms.

Conflict of interest

None of the authors is affiliated with Novego® enterprise who created HelpID. The novego company was neither involved in the planning nor in analyzing the results of the trial. For the purpose of the trial, participants received free vouchers which were paid for by Novego.

Contributors

Steffen Moritz,Johanna Schroder,Jan Philipp Klein and Tania M. Lincoln designed the study and wrote the protocol. Anja Fischer and Sonke Arltwere involved in the literature searches. Steffen Moritz, Johanna Schroder andJan Philipp Klein undertook the statistical analysis. All authors substantially contributed to and have approved the final manuscript.

Role of funding source

The study did not receive external funding and was carried out using existing budget resources of the department.

Acknowledgement

We would like to thank Helena Mayer, Katharina Kolbeck, Nora Ramdani, Thies Ludtke and Teresa Thoring for conducting the telephone interviews.

References

Alvarez-Jimenez, M., Alcazar-Corcoles, M.A., Gonzalez-Blanch, C., Bendall, S., McGorry, P.D., Gleeson, J.F., 2014. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr. Res. 156, 96-106.

S. Moritz etal./ Schizophrenia Research xxx (2016) xxx-xxx

Bechdolf, A., Klingberg, S., 2014. Psychotherapy of schizophrenia: not a problem of evidence, but a problem of implementation (Psychotherapie bei schizophrenen Storungen: Kein Evidenz-, sondern ein Implementierungsproblem). Psychiatr. Prax. 41,8-10.

Beck, A.T., Steer, R.A., 1993. Beck Depression Inventory Manual. Psychological Corporation, San Antonio.

Borm, G.F., Fransen,J., Lemmens, W.A., 2007. A simple sample size formula for analysis of covariance in randomized clinical trials. J. Clin. Epidemiol. 60,1234-1238.

Buckley, P.F., Miller, B.J., Lehrer, D.S., Castle, D.J., 2009. Psychiatric comorbidities and schizophrenia. Schizophr. Bull. 35, 383-402.

Burckhardt, K., 2012. Why it is Difficult to Distinguish between Negative and Depressive Symptomatology in Schizophrenia. University of Hamburg, Hamburg, Germany.

Byrne, R., Davies, L., Morrison, A.P., 2010. Priorities and preferences for the outcomes of treatment of psychosis: a service user perspective. Psychosis 3, 210-217.

Cuijpers, P., Donker, T.,Johansson, R., Mohr, D.C., Van Straten, A., Andersson, G., 2011. Selfguided psychological treatment for depressive symptoms: a meta-analysis. PLoS One 6, e21274.

Fischer, A., Schroder, J., Vettorazzi, E., Wolf, O.T., Pottgen, J., Lau, S., Heesen, C., Moritz, S., Gold, S.M., 2015. An online programme to reduce depression in patients with multiple sclerosis: a randomised controlled trial. Lancet Psychiatry 2, 217-223.

Freeman, D., Garety, P., 2014. Advances in understanding and treating persecutory delusions: a review. Soc. Psychiatry Psychiatr. Epidemiol. 49, 1179-1189.

Freeman, D., Garety, P., Fowler, D., Kuipers, E., Dunn, G., Bebbington, P., Hadley, C., 1998. The London-East Anglia randomized controlled trial of cognitive-behaviour therapy for psychosis. IV: Self-esteem and persecutory delusions. Br. J. Clin. Psychol. 37, 415-430.

Freeman, D., Garety, P.A., Bebbington, P.E., Smith, B., Rollinson, R., Fowler, D., Kuipers, E., Ray, K., Dunn, G., 2005. Psychological investigation of the structure of paranoia in a non-clinical population. Br. J. Psychiatry 186, 427-435.

Fusar-Poli, P., Nelson, B., Valmaggia, L., Yung, A.R., McGuire, P.K., 2014. Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis. Schizophr. Bull. 40, 120-131.

Garety, P.A., Kuipers, E., Fowler, D., Freeman, D., Bebbington, P.E., 2001. A cognitive model of the positive symptoms of psychosis. Psychol. Med. 31, 189-195.

Gilbody, S., Richards, D., Brealey, S., Hewitt, C., 2007. Screening for depression in medical settings with the patient health questionnaire (PHQ): a diagnostic meta-analysis. J. Gen. Intern. Med. 22, 1596-1602.

Hafner, H., Maurer, K., an der Heiden, W., 2013. ABC schizophrenia study: an overview of results since 1996. Soc. Psychiatry Psychiatr. Epidemiol. 48, 1021-1031.

Hautzinger, M., Brahler, M., 1993. Allgemeine Depressionsskala (ADS) [General Depression Scale]. Hogrefe, Gottingen.

Hor, K., Taylor, M., 2010. Suicide and schizophrenia: a systematic review of rates and risk factors. J. Psychopharmacol. 24, 81-90.

Jaspers, K., 1973. Allgemeine Psychopathologie [General Psychopathology]. Springer, Berlin (Germany).

Johansson, R., Andersson, G., 2012. Internet-based psychological treatments for depression. Expert. Rev. Neurother. 12, 861-870.

Karamatskos, E., Mulert, C., Lambert, M., Naber, D., 2012. Subjective well-being of patients with schizophrenia as a target of drug treatment. Curr. Pharm. Biotechnol. 13, 1490-1499.

Karow, A., Pajonk, F.G., Reimer, J., Hirdes, F., Osterwald, C., Naber, D., Moritz, S., 2008. The dilemma of insight into illness in schizophrenia: self- and expert-rated insight and quality of life. Eur. Arch. Psychiatry Clin. Neurosci. 258, 152-159.

Kasanin, J., 1933. The acute schizoaffective psychoses. Am. J. Psychiatr. 90, 97-126.

Kay, S.R., Opler, L.A., Lindenmayer, J.P., 1989. The positive and negative syndrome scale (PANSS): rationale and standardisation. Br. J. Psychiatry 155 (Suppl. 7), 59-67.

Kenward, M.G., Roger,J.H., 1997. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics 53, 983-997.

Kesting, M.L., Lincoln, T.M., 2013. The relevance of self-esteem and self-schemas to persecutory delusions: a systematic review. Compr. Psychiatry 54, 766-789.

Kishi, T., Iwata, N., 2014. Meta-analysis of noradrenergic and specific serotonergic antidepressant use in schizophrenia. Int. J. Neuropsychopharmacol. 17, 343-354.

Kishi, T., Hirota, T., Iwata, N., 2013. Add-on fluvoxamine treatment for schizophrenia: an updated meta-analysis of randomized controlled trials. Eur. Arch. Psychiatry Clin. Neurosci. 263, 633-641.

Kraepelin, E., 1899. Psychiatrie. Ein Lehrbuch fur Studierende Und Aerzte [Psychiatry. A Textbook for Students and Physicians]. J. A. Barth, Leipzig.

Kroenke, K., Spitzer, R.L.,Williams,J.B., 2001.The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606-613.

Kuck,J., Zisook, S., Moranville,J.T., Heaton, R.K., Braff, D.L., 1992. Negative symptomatology in schizophrenic outpatients. J. Nerv. Ment. Dis. 180, 510-515.

Leucht, S., Komossa, K., Rummel-Kluge, C., Corves, C., Hunger, H., Schmid, F., Asenjo Lobos, C., Schwarz, S., Davis, J.M., 2009. A meta-analysis of head-to-head comparisons of second-generation antipsychotics in the treatment of schizophrenia. Am. J.Psychiatr.166,152-163.

Lincoln, T.M., Lullmann, E., Rief, W., 2007. Correlates and long-term consequences of poor insight in patients with schizophrenia. A systematic review. Schizophr. Bull. 33, 1324-1342.

Lincoln, T.M., Peter, N., Schafer, M., Moritz, S., 2010. From stress to paranoia: an experimental investigation of the moderating and mediating role of reasoning biases. Psychol. Med. 40, 169-171.

Lincoln, T.M., Mobius, C., Huber, M.T., Nagel, M., Moritz, S., 2014. Frequency and correlates of maladaptive responses to paranoid thoughts in patients with psychosis compared to a population sample. Cogn. Neurodyn. 19, 509-526.

Moritz, S., Veckenstedt, R., Randjbar, S., Vitzthum, F., Karow, A., Lincoln, T.M., 2010. Course and determinants of self-esteem in people diagnosed with schizophrenia during psychiatric treatment. Psychosis 2, 144-153.

Moritz, S., Voigt, M., Kother, U., Leighton, L., Kjahili, B., Babur, Z., Jungclaussen, D., Veckenstedt, R., Grzella, K., 2014. Can virtual reality reduce reality distortion? Impact of performance feedback on symptom change in schizophrenia patients. J. Behav. Ther. Exp. Psychiatry 45, 267-271.

Moritz, S., Endlich, L., Mayer, H., Andreou, C., Ramdani, N., Petermann, F. & Balzan, R. P. (in press-a). The benefits of doubt: cognitive bias correction reduces hasty decisionmaking in schizophrenia. Cogn. Ther. Res.

Moritz, S., Berna, F., Jaeger, S., Westermann, S. & Nagel, M. (in press-b). The customer is always right? Subjective target symptoms and treatment preferences in patients with psychosis. Eur. Arch. Psychiatry Clin. Neurosci.

Naber, D., Karow, A., 2001. Good tolerability equals good results: the patient's perspective. Eur. Neuropsychopharmacol. 11 (Suppl. 4), S391-S396.

Peralta, V., Cuesta, M.J., 1994. Psychometric properties of the positive and negative syndrome scale (PANSS) in schizophrenia. Psychiatry Res. 53, 31-40.

Radloff, L.S., 1977. The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1, 385-401.

Richards, D., Richardson, T., 2012. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329-342.

Santor, D.A., Ascher-Svanum, H., Lindenmayer, J.P., Obenchain, R.L., 2007. Item response analysis of the positive and negative syndrome scale. BMC Psychiatry 7, 66.

Schneider, K., 1959. Clinical Psychopathology. Grune and Stratton, New York.

Schroder, J., Bruckner, K., Fischer, A., Lindenau, M., Kother, U., Vettorazzi, E., Moritz, S., 2014. Efficacy of a psychological online intervention for depression in people with epilepsy: a randomized controlled trial. Epilepsia 55, 2069-2076.

Shafran, R., Clark, D.M., Fairburn, C.G., Arntz, A., Barlow, D.H., Ehlers, A., Freeston, M., Garety, P.A., Hollon, S.D., Ost, L.G., Salkovskis, P.M., Williams, J.M.G., Wilson, G.T., 2009. Mind the gap: improving the dissemination of CBT. Behav. Res. Ther. 47, 902-909.

Sheehan, D.V., Lecrubier, Y., Sheehan, K.H., Amorim, P., Janavs, J., Weiller, E., Hergueta, T., Baker, R., Dunbar, G.C., 1998. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 59 (Suppl. 20), 22-33.

Spek, V., Cuijpers, P., Nyklicek, I., Riper, H., Keyzer, J., Pop, V., 2007. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol. Med. 37, 319-328.

Sterk, B., Winter van Rossum, I., Muis, M., de Haan, L., 2013. Priorities, satisfaction and treatment goals in psychosis patients: an online consumer's survey. Pharmacopsychiatry 46, 88-93.

Suzuki, T., 2011.Which rating scales are regarded as the standard' in clinical trials for schizophrenia? A critical review. Psychopharmacol. Bull. 44, 18-31.

Van der Gaag, M., Hoffman, T., Remijsen, M., Hijman, R., de Haan, L., van Meijel, B., van Harten, P.N., Valmaggia, L., de Hert, M., Cuijpers, A., Wiersma, D., 2006. The five-factor model of the positive and negative syndrome scale II: a ten-fold crossvalidation of a revised model. Schizophr. Res. 85, 280-287.

Walker, C., 1991. Delusion: what did jaspers really say? Br. J. Psychiatry 159 (Suppl. 14), 94-103.

Wykes, T., Steel, C., Everitt, B., Tarrier, N., 2008. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr. Bull. 34, 523-537.

Original Article

Facilitating the Delivery of Cognitive Remediation in First-Episode Psychosis

Pilot Study of a Home-Delivered Web-Based Intervention

Bernardo Melo Moura, MD,30f Alessia Avila, PsyD, ff Ines Chendo, MD,30f Patricia Frade, MD,30 Rita Barandas, MD,30f Joao Vian, MD,§ Marta Croca, MD,30f Alice Roberto, MD,f Carolina Almeida, MD,§ Filipa Antunes, MD,30 Ludgero Linhares, MD,30 Joana Crawford, MD,30 Carla Ferreira, MD,30 Jennifer Santos, MD,30 Manuela Abreu, MD,30 Pedro Levy, MD,30 Maria Luisa Figueira, MD, PhD, f and Tiago Mendes, MSc30f

Abstract: We explored the adherence to a home-delivered, computer-based, cognitive remediation protocol in a first-episode psychosis outpatient cohort. Seventeen patients underwent a cognitive training protocol for 6 months using an online platform accessible from their home under the supervision of a qualified neuropsychologist. Neuropsychological, psychopathological, and functional data were collected at baseline and postintervention, whereas qualitative appraisal of the intervention was assessed monthly. Overall, participants' evaluation of the program was positive. This was reflected in a good adherence rate with 12 (70%) of 17 patients completing 80% of the prescribed sessions. Exploratory analysis revealed significant improvements in sustained attention (p = 0.020) and verbal memory (p = 0.018). A decrease in negative symptoms and an improvement on the Clinical Global Impression were also found (p = 0.009). We believe these are encouraging results to further explore the adopted delivery approach, which could facilitate access to cognitive training earlier and to a larger group of patients.

Key Words: Cognitive remediation, first-episode psychosis, Web based, adherence, home delivered

(JNew Ment Dis 2019;207: 951-957)

It is now well established that psychotic disorders are consistently associated with impairments in neurocognition (Bora and Pantelis, 2015; Heinrichs and Zakzanis, 1998). Deficits are present onawide variety of domains, with meta-analytical data indicating verbal memory and processing speed as the cognitive skills carrying the highest deficits (Mesholam-Gately et al., 2009). Considering that cognitive impairments are an important predictor of low psychosocial functioning (Green, 2006), deficits in this domain seem to play a key role in the poor prognosis of psychotic disorders and in the illness burden on patients' lives.

in the lack of effective pharmacological treatments, cognitive remediation (CR) has established itself as an effective intervention for cognitive impairment in chronic patients (Wykes et al., 2011). Rooted in a neurodevelopmental approach to the illness, CR is defined as “a behavioral training-based intervention that aims to improve cognitive processes (attention, memory, executive function, social cognition, or meta-cognition) with the goal of durability and generalization” (Cognitive Remediation Experts Workshop [CREW], Florence, April 2010). Meta-analytical evidence from 40 controlled studies on CR in chronic schizophrenia has demonstrated that CR provides durable benefits in global cognition and functioning—effect sizes 0.45 and 0.42, respectively—against any control group (Wykes et al., 2011). Despite a growing emphasis on the need to implement CR interventions earlier in the illness progression to prevent the worsening and chronicization of cognitive deficits (Bora and Murray, 2014), there is a paucity of studies focusing on first-episode psychosis (FEP). So far, research on this population indicated acceptability and benefit in the short and longer term on both cognitive and functional outcomes, particularly when CR is combined with other types of interventions (Revell et al., 2015).

Although CR is implemented through different methods, the main differences between programs concern the level of therapist support (thus the costs of implementation) and the training delivery, varying from face-to-face supervised sessions to group setting and homebased programs (Wykes and Spaulding, 2011). Most CR interventions propose an intensive training program, encompassing weekly face-to-face sessions for several months (Revell et al., 2015). Considered as such, the delivery of CR within public early intervention services could prove logistically extremely demanding for both patients and mental health institutions. In addition, given that patients' disengagement from services is one of the major hurdles in this field (Doyle et al., 2014; Stowkowy et al., 2012), efforts should be made to maximize adherence while designing new interventions, for instance, by taking into account characteristics such as setting and location (Lucksted et al., 2015). Supervised Web-based programs, accessible independently by the patients from familiar settings of their choice (from here on referred to as “home delivered” for brevity), might represent a feasible solution to facilitate the delivery and accessibility of CR interventions. In fact, people with severe mental disorders, including FEP, seem to use information technology comparably to the general population (Abdel-Baki et al., 2017; Ennis et al., 2012; Thomas et al., 2017). In addition, evidence from computer-assisted CR interventions indicates high acceptability and positive outcomes on a wide range of neurocognitive and social-cognitive domains (Berry et al., 2016; Naslund et al., 2015).

To our knowledge, no study has investigated the use of a home-delivered, supervised approach in FEP. This format, conjugating independent sessions and face-to-face supervision, might be of a particular interest in a public health setting because it combines potential therapeutic gains with a reduced economic and logistic burden. In the present study, we aim at investigating the adherence to a Web-based, home-delivered, supervised CR intervention for patients with a FEP. The chosen training protocol can be ascribed to the “bottom-up” approaches and aims at strengthening core cognitive domains through drill and practice exercising.

The Journal of Nervous and Mental Disease Volume 207, Number 11, November 2019


www.jonmd.com I 951


METHODS

Participants

Participants were recruited from an outpatient clinic for early psychosis (PROFIP) in the Department of Psychiatry and Mental Health of Santa Maria University Hospital, North Lisbon Hospital Centre. Data were collected between November 2013 and May 2016. The PROFIP program is directed to individuals who experienced a FEP within the previous 6 months and have not yet received treatment. Eligibility criteria for the study enrollment were a) diagnosis of FEP for 5 years or less, b) age between 18 and 35 years at the time of enrollment, c) being symptomatically stable on medications, and d) have access to a computer and being an independent user. The set exclusion criteria were a) illicit substance use in the past 6 months, b) a diagnosis of affective psychosis, c) clinical instability in the past 6 months, and d) presence of an underlying neurologic condition affecting cognition. Of the 102 patients listed on the early intervention program, only a minority was approached for the study. Many factors contributed to reduction of the number of potential participants: 43 patients stopped attending the service, 14 were active cannabis users, 10 were clinically unstable, 9 had a current diagnosis of bipolar disorder, and 9 had a diagnosis of FEP for longer than 5 years. Finally, 17 patients were invited to participate in the study. Descriptive demographic characteristics are summarized in Table 1.

Design

Because of our aim of assessing adherence to a specific implementation approach and because of the low availability of eligible patients, we designed a pilot study delivering the intervention to all the participants in a longitudinal, repeated-measures fashion. The neuropsychologist who supervised the CR program and completed the cognitive assessments was blinded to the pharmacological intervention and all other assessments. Clinical and functional assessments were performed by a senior psychiatrist blinded to neuropsychological status and to the CR performance. Clinical Global Impression (CGI) was rated by a second psychiatrist blinded to all other assessments and CR performance. The intervention was offered as an add-on to treatment as usual, which included pharmacotherapy, psychiatry sessions, psychoeducation, and family interventions.

Measures

Our main outcome measure was adherence, measured as the ratio between the total number of hours of training completed by each participant and the total number of hours prescribed. Other measures informing adherence included the following:

TABLE 1. Baseline Demographic and Clinical Characteristics of the Sample

Mean or %

SD

Min.

Max.

Age, yrs

23.6

3.9

18

31

Sex (male)

80

Education, yrs

12.7

2.9

9

16

Chlorpromazine equivalents, mg

172.5

129.3

50

450

Months of illness

24.6

15.7

9

60

952 I www.jonmd.com

A dropout was considered whenever a participant interrupted the training either actively (participant communicates decision to not continue) or passively (2 weeks without accessing the program) or presented a clinical relapse. Reasons for discontinuation were registered.

Neuropsychological, clinical, and functional assessments were performed at baseline and postintervention to inform exploratory analysis. The specific measures adopted are summarized below. Neuropsychological assessment was organized into five theoretically derived cognitive domains:

Subjective memory complaints (SMCs) were also collected using the homonymous scale (Schmand et al., 1996; Portuguese version validated by Gino et al., 2008). This is composed of 10 individual items concerning difficulties in daily life memory tasks, giving a total score from zero (absence of complaints) to 21 (maximal complaints score). The SMC scale is brief compared with other SMC scales, and its items content has been considered representative of common memory complaints (Schmand et al., 1996).

Clinical symptoms were evaluated using the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987) and Clinical Global Impression-Severity (CGI-S) scale (Brissos et al., 2012). Social functioning was assessed using the Global Assessment of Functioning (Endicott et al., 1976) and the Personal and Social Performance scale (Morosini et al., 2000; Portuguese version by Brissos et al., 2012).

© 2019 Wolters Kluwer Health, Inc. All rights reserved.

Procedures

The study was approved by the hospital ethics committee, and all the enrolled participants gave their written informed consent before undergoing any procedure.

A comprehensive neuropsychological, clinical, and functional assessment was administered at baseline and postintervention.

An experienced neuropsychologist was responsible for the individual supervision of the participants' training. This included a) an introductory face-to-face session; b) monthly appointments with participants and their carers at the clinical center to collect subjective appraisals, address eventual issues, and provide feedback on the training progression; c) weekly remote supervision of the training progress and assignment of the exercises for the subsequent week; and d) permanent remote availability for troubleshooting. During the first introductory session, the patients were familiarized with the interface and given their access credentials; they were then trained to use the software using a demo training session. The same set of three exercises (one per cognitive domain) was prescribed to every participant for the first week of training. At the end of each week, the exercises were renewed to avoid habituation and disinterest. After the fourth week of training, the prescribed exercises became tailored to the individual performance. More precisely, exercises previously prescribed, but still with a margin of progression, would be prescribed again starting at level N-1 of the maximum level achieved by the patient; exercises where progression reached the top were dropped, and new ones were inserted by the supervising neuropsychologist.

Moreover, the participants were asked to perform the training in a quiet and silent place to avoid any distraction, including listening to music, and not to interrupt the sessions. Patients were also requested to send the supervising neuropsychologist an e-mail after completing the first session to provide feedback on the experience and discuss any issue. This first contact was made compulsory for the training to continue, and the neuropsychologist would proactively seek contact with the participant if the e-mail was not delivered.

Intervention

The CR intervention was delivered using the COGWEB program, a Web-based software that allows carrying out a personalized training from a personal computer at the participant's home (or any other chosen setting outside the hospital), under clinical supervision (Cruz et al., 2013).

The program comprises a series of exercises in a computerized game format (Fig. 1) organized primarily around a specific cognitive function. In this study, the training focused primarily on the three most affected domains in this population (also confirmed in our sample): memory, attention, and executive functions (Mesholam-Gately et al., 2009). A pool of 21 different exercises was used. The COGWEB program allows the remote adjustments of several parameters, namely, a) duration of each daily session, b) number of sessions per week, c) time of the day (morning or afternoon) for training, d) the initial level of difficulty, and e) the duration for each exercise during a session (Cruz et al., 2013). The software algorithms provide an automatic adjustment of the level of difficulty according to the participant's performance, following an error-less learning approach. Progression is achieved through increasing the number of items per level and their complexity or by reducing the time interval between stimuli. To prevent confounding learning effects and boost motivation, the exercises use random, nonsequential stimuli. Progress graphs and diagrams reflecting participants' performance (i.e., correct answers and errors, levels completed, global training time) are prompted by the software and were used to assess adherence and provide participants with visual feedback on their improvements in the training during monthly monitoring sessions.

Participants were asked to perform 30 to 35 minutes of daily sessions during weekdays, preferably in the morning, and pause at

weekends. Each session always consisted of three different exercises. The intervention was offered for 6 months.

Data Analyses

Adherence was calculated per each individual by using the following formula: number of total training hours performed/number of total training hours prescribedx100. Normal distribution of neuropsychological, clinical, and functional outcome data was tested through the Kolmogorov-Smirnov test. Changes before and after CR were explored using paired-sample t-test. Because this was a pilot study with a small sample size and the analysis was only exploratory, type I error was not adjusted for number of comparisons and was kept at a < 0.05 level for each comparison.

RESULTS

Adherence

Of the 17 participants included in the study, 12 completed the training protocol and 5 were considered dropouts. Factors contributing to the attrition rate were loss of interest (three participants), clinical instability (one participant), and changes in professional status leading to lack of available time (one participant).

Figure 2 shows the rate of prescribed training time completed by each participant on each week and the overall mean completion rate, represented by the line in bold (dropouts are also included). Participants spent an average of 43 hours on training, thus adhering to around 80% of the prescribed training time (SD = 24.2%). As shown in the graph, a few participants spent more time in the exercises than prescribed; therefore, their completion rate is reported as higher than 100%.

On average, participants spent 44.8% of time doing exercises focusing on attention (SD = 4.1%), 36.7% of time on memory training exercises (SD = 4.5%), and 18.6% on executive function training exercises (SD = 7.4%). Overall, participants improved their

160



1   2   3   4   5   6   7   8   9   10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Weeks

FIGURE 2. Participants' weekly completion rate through the training protocol (soft gray lines represent individual participants, and the line in bold displays the average rate).

performance in each training domain. In particular, the average progression rates per domain were 69% for attention, 42% for memory, and 122% for executive functioning exercises.

As an additional measure of adherence, we report the average number of face-to-face contacts with the supervising neuropsychologist. The study implementation involved seven face-to-face visits throughout the intervention (one introductory and one at the end of each month). On average, participants attended five face-to-face visits. Interestingly though, the participants who fully adhered to the treatment came to all the visits, whereas those who dropped out mostly attended only the first one or two visits; in total, 90 visits were performed.

Exploratory Analysis of Neuropsychological, Clinical, and Functional Outcomes

Table 2 presents scores for the neuropsychological, clinical, and functional outcomes at baseline and postintervention. There was an overall improvement in neuropsychological measures, with statistical significance for sustained attention (Toulouse-Pieron dispersion index; p = 0.020) and verbal memory (CVLT-LDFR; p = 0.018). Cognitive flexibility, as reflected by the score on perseverative errors in the WCST, also registered a trend for improvement approaching significance (p = 0.053).

Finally, a significant improvement in SMCs was found after CR (p = 0.022).

Concerning clinical symptoms, a decrease on the total PANSS score was found, mainly driven by a strongly significant drop in the negative symptoms domain (p = 0.014). A significant improvement (p = 0.009) was also registered in symptom severity as assessed by an independent rater on the CGI (severity domain). Changes in functional outcome measures did not reach statistical significance.

During the study, baseline antipsychotic medication remained unchanged for all the participants who completed the training protocol (mean chlorpromazine equivalents = 172.5 mg, SD = 129.3 mg).

TABLE 2. Analysis of the Results Comparing Conditions at Baseline and Postintervention

Baseline

Postintervention

n

pa

Mean (SD) Score

Mean (SD) Score

Toulouse-Pieron (dispersion index)

18.8 (18.3)

9.9 (8.9)

12

0.020*

California Verbal Learning Test (LDFR)

10.0 (4.1)

12.1 (2.9)

11

0.018*

Trail Making Test B (time)

95.9 (34.7)

100.1 (95.3)

11

0.854

Rey Complex Figure (delayed visual memory)

22.4 (4.8)

23.3 (7.0)

12

0.580

WCST (perseverative errors)

8.2 (5.2)

6.2 (2.5)

11

0.053

Digit Span Backwards

4.0 (1.1)

4.2 (0.7)

11

0.339

SMC (total score)

5.17(3.1)

3.58(1.9)

12

0.022*

Semantic Verbal Fluency (total score)

19.2 (5.6)

20.9 (6.7)

12

0.270

PANSS positive symptom score

9.6 (2.5)

8.6 (2.0)

9

0.063

PANSS negative symptom score

24.3 (5.7)

19.6 (5.8)

9

0.014*

GAF

67.9 (17.9)

77.8 (16.0)

8

0.152

CGI-S

3.5 (1.3)

2.7 (1.1)

8

0.009**

PSP

63.3 (16.9)

76.0 (18.2)

9

0.073

GAF indicates Global Assessment of Functioning; PSP, Personal and Social Performance. Paired t-test.

*p < 0.05.

**p <0.01.


954 I www.jonmd.com


© 2019 Wolters Kluwer Health, Inc. All rights reserved.


Acceptability and Satisfaction

Semistructured interviews were carried out during the monthly visits by the supervising neuropsychologist to assess acceptability and satisfaction. The interviews consistently assessed four areas: a) motivation and satisfaction, b) mental fatigue, c) acceptability of training characteristics, and d) physical complaints. Most of the participants expressed an overall positive appraisal of the training program; moreover, half of them referred perceived improvement in memory and concentration, as well as a higher participation in daily life activities after 2 or 3 months of training. About one third of participants (33.3%) mentioned feeling “mental fatigue” related with the exercises at some point of the protocol; however, this seemed to be a fleeting concern, as most subjects did not report the symptom again by the end of the training. The training's level of difficulty and the sessions' duration were both considered adequate, although four participants (33.3%) described some of the tasks as “boring.” No participants reported physical complaints.

Of the three participants who discontinued the training due to loss of interest, two mentioned mental fatigue as a reason for abandoning the study, and one expressed a preference toward direct supervision while training.

When present, accompanying carers did not report any negative consequences of taking part in the study, and most confirmed the perceived improvements of their relatives.

DISCUSSION

Adherence and Acceptability

Despite CR's established validity in chronic patients, relatively little evidence has been collected in FEP cohorts. Moreover, no studies investigated the use of home-delivered supervised programs in this population. Results from the present study showed a satisfactory adherence rate, with 12 (70%) of the 17 participants completing the protocol and adhering to over 80% of the prescribed sessions. The attrition rate in our study is in line with the figure reported in other studies conducted so far, delivering CR to patients with FEP (Revell et al., 2015). A study by Dillon et al. (2016) using a sample of patients with established schizophrenia undergoing an analogous online CR program showed a higher attrition rate. This might be explained by sample characteristics (higher illness duration) and/or by differences in the chosen implementation (lower support). Nevertheless, we believe the present study adopted several measures to sustain adherence, mostly by strengthening the therapeutic relationship with the supervising neuropsychologist: a compulsory feedback from the participant after the first session, regular face-to-face meetings, and a permanent availability for troubleshooting. The relationship with the supervising professional is a key ingredient of CR programs and ultimately what makes them a therapeutic tool rather than a common “brain training.” The suggested implementation values the role of the professional while lifting the logistic burden of numerous face-to-face sessions and placing trust on the patients' autonomy. Another factor that might have positively affected adherence was a relatively high diversity of training tasks (at least 12 already in the first month), which might have increased interest. Finally, it is important to highlight that the study received the support and endorsement of the consultant psychiatrist and was conducted in synergy with the clinical team.

We explored acceptability through semistructured interviews administered monthly during a face-to-face session with the participants and their carers. Most of the participants expressed satisfaction and valued taking part in the intervention. In particular, patients reported a subjective feeling of improvement in memory and concentration as well as of higher involvement in everyday tasks: these impressions were also supported by an improvement in scores on the SMC scale. Interviews with participants and their carers also revealed that CR was valued by families and indicated that during the study, some carers became more involved in the recovery process of their affected relative. However, because the involvement of carers and its effect on adherence were not systematically assessed, conclusions should be drawn carefully. We believe these aspects should receive more attention in future studies, and the role of carers should be evaluated by assessing their engagement in treatment, their attitudes and beliefs toward cognitive deficits, and their expectations toward CR.

Overall, training characteristics were considered adequate despite a few participants indicating some exercises as tedious. These remarks might reflect the importance of creating more engaging digital interfaces to sustain motivation, particularly when targeting a young population. Mental fatigue was reported in some instances; however, this symptom tended to disappear with the progression of the protocol, suggesting that participants were more able to tolerate the intensity of the sessions as the training progressed. In fact, despite our choice of a rather long training protocol, the analysis of the total training time performed versus the one prescribed per each individual showed no significant changes in the pattern of adherence along the 6 months of training. This seems to suggest that a longer protocol did not affect changes in adherence, but rather, the participant's “training style” remained consistent toward the program.

Despite all, the implementation angle that we chose might not be suitable for all patients with FEP. In one instance in fact, a participant dropped out due to a preference for face-to-face sessions, and it can be speculated that providing closer monitoring might improve overall adherence in some instances. Ideally, interventions should be tailored to the individual preference, and future studies should consider different delivery and implementation characteristics.

By the end of the training, all participants improved their performance on each task. However, when considering training domains, the relative progression seemed to be smaller for exercises tackling memory rather than attention and executive functions. This might be due to memory being an area of major impairment in this cohort, where progress might be more difficult to generate.

Considered together, these findings indicate that granting participants more autonomy in accessing CR interventions independently, from familiar settings of their choice, does not reduce adherence significantly, although it might facilitate the delivery of the intervention within resource-limited services. These findings seem to be in line with those from Biagianti et al. (2017), which shows a similar adherence rate between CR delivered remotely and in a laboratory setting in a sample of patients with chronic schizophrenia.

Changes in Outcomes

The performed exploratory analyses indicated that changes in cognitive and clinical outcomes might have occurred after the training. Indeed, participants showed a significant improvement in sustained attention and episodic verbal memory, as well as a trend for improvement in executive functions. These findings are consistent with the results of previously published studies on CR interventions in patients with psychosis (Eack et al., 2009; Fisher et al., 2015; Lee et al., 2013; Wykes et al., 2007). Because of the lack of a control group in the design, the analyses remain of a speculative nature, and no causal effect can be inferred between the interventions and the changes in outcomes.

In the present study, memory complaints decreased after CR. However, because the neuropsychologist who administered the SMC scale was not blind to the intervention, the reported improvement might have been confounded by a social desirability effect. Nonetheless, the perceived memory enhancement is in line with the overall improvement detected in cognitive domains and was supported by the carers' impressions.

Participants showed a significant improvement in negative symptoms after training. This is in line with other studies, and meta-analytical evidence, on patients with psychosis, indicating small to moderate effect of CR on negative symptoms, with more methodologically sound studies reporting stronger changes (Cella et al., 2017; Ventura et al., 2017). Yet, the absence of a control group limits the appreciation of these findings.

Social functioning did not show a significant improvement after the intervention. This is in line with previous CR trials using a bottom-up approach and suggests that protocols aiming at improving functioning as a core outcome should integrate training social cognition skills to help generalize cognitive improvement to everyday tasks (Wykes and Spaulding, 2011).

Limitations

Several limitations in the present study should be considered. First and foremost, the reduced sample size and the lack of a control group strongly limit the appreciation of the changes in outcomes detected through exploratory analysis. However, as described above, the sample included in the study represented the total of available participants in the service, meeting the chosen inclusion criteria at the time. In addition, because the study aimed at assessing mainly adherence and acceptability of the novel characteristics of the intervention, a control group was not held essential.

The sample's heterogeneity regarding the duration of the disorder at the time of enrollment might also potentially affect the interpretation of the results. A different illness duration—even if within a 5-year window—might play a role in affecting both performance and progression on cognitive training.

Moreover, the absence of blinding for the neuropsychological assessment and the monitoring of the intervention might have influenced the participants, particularly during the evaluation of perceived improvements and the collection of participants' appraisals.

Finally, the qualitative data of patients' acceptability and appraisals could have been collected more systematically, particularly in regard to the influence of the neuropsychologist's input and the involvement of carers.

CONCLUSIONS AND RECOMMENDATIONS

We conclude that CR interventions in FEP can be implemented using protocols that allow independent access to the training through a Web-based software in the patient's own environment. The study also suggests that the combination of independent sessions with a regular supervision by a professional might improve adherence to the intervention. In our opinion, this type of implementation might turn CR interventions logistically and economically more feasible for public service: if proven effective, in the near future, CR could be considered as a treatment option systematically available for patients with FEP (Cella et al., 2015; Wykes and Spaulding, 2011).

Drawing on both the strengths and limitations of the present study, some recommendations for future research can be drawn. First, we believe we have identified a good balance between independent and supervised sessions and that the efficacy of this specific implementation should be tested in a methodologically sound clinical trial. Within a putative protocol, exclusion criteria regarding drug use should be narrowed by only leaving out participants who meet criteria for drug abuse in the previous 6 months (instead of any use)—this way improving recruitment rates and external validity. Data on drug consumption should be then systematically collected and the results controlled for confounding effects. Second, the duration of the protocol should also be investigated. In our pilot study, the longer duration of the intervention did not seem to negatively affect adherence. However, it should be advisable to run interim analyses halfway through the end of the training (3 months) to explore how changes in outcomes behave throughout time and inform the design accordingly. Third, based on our preliminary analyses and in line with the literature, more data should be collected on negative symptoms to explore whether patients

956 I www.jonmd.com

with a more severe profile benefit the most from the intervention— thus informing potential sample stratification. Finally, we believe that two variables deserve a more consistent collection of information: the relationship with the supervising neuropsychologist and the level of involvement of carers. These aspects are likely to be central in maintaining adherence but are underinvestigated in the available literature. As supplementary materials, we have provided a schematic flow diagram of a putative protocol (Supplementary Fig. 1, Supplemental Digital Content 1, http://links.lww.com/JNMD/A77), including a draft of the study visit schedule (Supplementary Fig. 2, Supplemental Digital Content 2, http://links.lww.com/JNMD/A78).

ACKNOWLEDGMENTS

The authors would like to thank Professor Joana Pais at Neuroinova for providing the COG WEB platform for this study and Mrs Ana Fernandes at the Faculty of Medicine for logistical support.

DISCLOSURE

The present work was funded by the Portuguese National Healthcare Service.

B.M.M. and A.A. benefitted from grants from Fundagdo para a Ciencia e Tecnologia PD/BD/128404/2017 and SFRH/BD/115916/ 2016, respectively.

The authors declare no conflict of interest.

REFERENCES

Abdel-Baki A, Lal S, D-Charron O, Stip E, Kara N (2017) Understanding access and use of technology among youth with first-episode psychosis to inform the development of technology-enabled therapeutic interventions. Early Interv Psychiatry. 11:72-76.

Arbuthnott K, Frank J (2000) Trail Making Test, part B as a measure of executive control: Validation using a set-switching paradigm. J Clin Exp Neuropsychol. 22: 518-528.

Baeta E (2002) Bateria para avaliagao neuropsicologica de adultos com epilepsia. Psicologia. 16:79-96.

Berry N, Lobban F, Emsley R, Bucci S (2016) Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: A systematic review. J Med Internet Res. 18:el21.

Biagianti B, Fisher M, Howard L, Rowlands A, Vinogradov S, Woolley J (2017) Feasibility and preliminary efficacy of remotely delivering cognitive training to people with schizophrenia using tablets. Schizophr Res Cogn. 10:7-14.

Bora E, Murray RM (2014) Meta-analysis of cognitive deficits in ultra-high risk to psychosis and first-episode psychosis: Do the cognitive deficits progress over, or after, the onset of psychosis? Schizophr Bull. 40:744-755.

Bora E, Pantelis C (2015) Meta-analysis of cognitive impairment in first-episode bipolar disorder: Comparison with first-episode schizophrenia and healthy controls. Schizophr Bull. 41 :lO95-1104.

Brissos S, Palhava F, Marques JG, Mexia S, Carmo AL, Carvalho M, Dias C, Franco JD, Mendes R, Zuzarte P, Carita AI, Molodynski A, Figueira ML (2012) The Portuguese version of the Personal and Social Performance scale (PSP): Reliability, validity, and relationship with cognitive measures in hospitalized and community schizophrenia patients. Soc Psychiatry Psychiatr Epidemiol. 47:1077-1086.

Cella M, Preti A, Edwards C, Dow T, Wykes T (2017) Cognitive remediation for negative symptoms of schizophrenia: A network meta-analysis. Clin Psychol Rev. 52: 43-51.

Cella M, Reeder C, Wykes T (2015) Cognitive remediation in schizophrenia—Now it is really getting personal. Curr Opin Behav Sci. 4:147-151.

Cruz VT, Pais J, Bento V, Mateus C, Colunas M, Alves I, Coutinho P, Rocha NP (2013) A rehabilitation tool designed for intensive web-based cognitive training: Description and usability study. JMIR Res Protoc. 2:e59.

Delis DC, Kramer JH, Kaplan E, Thompkins BA (1987) CVLT: California Verbal Learning Test-Adult Version: Adult version. In Manual. San Antonio, TX: Psychological Corporation.

© 2019 Wolters Kluwer Health, Inc. All rights reserved.

Dillon R, Hargreaves A, Anderson-Schmidt H, Castorina M, Corvin A, Fitzmaurice B, Robertson I, Donohoe G (2016) Adherence to a low-support cognitive remediation training program for psychosis. J Nerv Ment Dis. 204:741-745.

Doyle R, Turner N, Fanning F, Brennan D, Renwick L, Lawlor E, Clarke M (2014) First-episode psychosis and disengagement from treatment: A systematic review. Psychiatr Serv. 65:603-611.

Eack SM, Greenwald DP, Hogarty SS, Cooley SJ, DiBarry AL, Montrose DM, Keshavan MS (2009) Cognitive enhancement therapy for early-course schizophrenia: Effects of a two-year randomized controlledtrial. Psychiatr Serv. 60:1468-1476.

Endicott J, Spitzer RL, Fleiss JL, Cohen J (1976) The global assessment scale. A procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry. 33:766-771.

Ennis L, Rose D, Denis M, Pandit N, Wykes T (2012) Can't surf, won't surf: The digital divide in mental health. JMent Health. 21:395 403.

Fisher M, Loewy R, Carter C, Lee A, Ragland JD, Niendam T, Schlosser D, Pham L, Miskovich T, Vinogradov S (2015) Neuroplasticity-based auditory training via laptop computer improves cognition in young individuals with recent onset schizophrenia. Schizophr Bull. 41:250-258.

Gino S, Guerreiro M, Garcia C (2008) Subjective memory complaints. In Tests and scales in dementia (2nd ed). Lisbon, Portugal: Group forthe Study ofBrain Aging and Dementia.

Green MF (2006) Cognitive impairment and functional outcome in schizophrenia and bipolar disorder. J Clin Psychiatry. 67:3-8.

Heaton RK, Chelune GJ, Curtiss G, Kay GG, Talley JL (1993) Wisconsin Card Sorting Test. Odessa, FL: Psychological Assessment Resources, Inc.

Heinrichs RW, Zakzanis KK(1998) Neurocognitive deficit in schizophrenia: A quantitative review of the evidence. Neuropsychology. 12:426-445.

Kay SR, Fiszbein A, Opler LA (1987) The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull. 13:261-276.

Kramer JH, Yaffe K, Lengenfelder J, Delis DC (2003) Age and gender interactions on verbal memory performance. JInt Neuropsychol Soc. 9:97-102.

Lee RS, Redoblado-Hodge MA, Naismith SL, Hermens DF, Porter MA, Hickie IB (2013) Cognitive remediation improves memory and psychosocial functioning in first-episode psychiatric out-patients. Psychol Med. 43:1161-1173.

Lezak MD, Howieson DB, Loring DW (2004) Neuropsychological assessment (4th ed). Oxford: Oxford University Press.

Lucksted A, Essock SM, Stevenson J, Mendon SJ, Nossel IR, Goldman HH, Goldstein AB, Dixon LB (2015) Client views of engagement in the RAISE Connection Program for early psychosis recovery. Psychiatr Serv. 66:699-704.

McDowell BD, Bayless JD, Moser DJ, Meyers JE, Paulsen JS (2004) Concordance between the CVLT and the WMS-III word lists test. Arch Clin Neuropsychol. 19:319-324.

Mesholam-Gately RI, Giuliano AJ, Goff KP, Faraone SV Seidman LJ (2009) Neurocognition in first-episode schizophrenia: A meta-analytic review. Neuropsychology. 23: 315-336.

Morosini PL, Magliano L, Brambilla L, Ugolini S, Pioli R (2000) Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social functioning. Acta Psychiatr Scand. 101:323-329.

Naslund JA, Marsch LA, McHugo GJ, Bartels SJ (2015) Emerging mHealth and eHealth interventions for serious mental illness: A review of the literature. J Ment Health. 24:321-332.

Osterrieth PA (1944) Le test de copie d'une figure complexe; Conteribution a l'etude de la perception et de la memoire [Test of copying a complex figure; Contribution to the study of perception and memory]. Arch Psychol. 30: 206-356.

Perianez JA, Rios-Lago M, Rodriguez-Sanchez JM, Adrover-Roig D, Sanchez-Cubillo I, Crespo-Facorro B, Quemada JI, Barcelo F (2007) Trail Making Test in traumatic brain injury, schizophrenia, and normal ageing: Sample comparisons and normative data. Arch Clin Neuropsychol. 22: 433-447.

Reitan RM, Wolfson D (1985) The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press.

Revell ER, Neill JC, Harte M, Khan Z, Drake RJ (2015) A systematic review and meta-analysis of cognitive remediation in early schizophrenia. Schizophr Res. 168:213-222.

Rocha AM, Coelho MH (1988) Manual do teste de figuras complexas de Andre Rey. Lisboa, Portugal: Cegoc-TEA, Lda.

Schmand B, Jonker C, Hooijer C, Lindeboom J (1996) Subjective memory complaints may announce dementia. Neurology. 46:121-125.

Spreen O, Strauss E (1998) A compendium of neuropsychological tests, administration, norms and commentary (2nd ed). Oxford: Oxford University Press.

Stowkowy J, Addington D, Liu L, Hollowell B, Addington J (2012) Predictors of disengagement from treatment in an early psychosis program. Schizophr Res. 136:7-12.

Thomas N, Foley F, Lindblom K, Lee S (2017) Are people with severe mental illness ready for online interventions? Access and use of the Internet in Australian mental health service users. Australas Psychiatry. 25:257-261.

Ventura J, Subotnik KL, Gretchen-Doorly D, Casaus L, Boucher M, Medalia A, Bell MD, Hellemann GS, Nuechterlein KH (2017) Cognitive remediation can improve negative symptoms and social functioning in first-episode schizophrenia: A randomized controlled trial. Schizophr Res. 203:24-31.

Wechsler D (1945) Wechsler memory scale. San Antonio, TX: Psychological Corporation.

Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P (2011) A meta-analysis of cognitive remediation for schizophrenia: Methodology and effect sizes. Am J Psychiatry. 168:472^85.

Wykes T, Newton E, Landau S, Rice C, Thompson N, Frangou S (2007) Cognitive remediation therapy (CRT) for young early onset patients with schizophrenia: An exploratory randomized controlled trial. Schizophr Res. 94: 221-230.

Wykes T, Spaulding WD (2011) Thinking about the future cognitive remediation therapy—What works and could we do better? Schizophr Bull. 37(suppl 2): S80-S90.


Contents lists available at ScienceDirect

Schizophrenia Research: Cognition

journal homepage: http://www.schizrescognition.com/


A novel, online social cognitive training program for young adults with schizophrenia: A pilot study                                                 u"

Mor Nahum a,b,31 32, Melissa Fisherc,d, Rachel Loewyc, Gina Poelkec,d, Joseph Ventura e, Keith H. Nuechterlein e,f, Christine I. Hooker g, Michael F. Green e,h, Michael M. Merzenich a, Sophia Vinogradovc,d

a Posit Science, San Francisco, CA, USA

b Department of Optometry, University of California, Berkeley, CA, USA

c Department of Psychiatry, University of California, San Francisco, CA, USA

d San Francisco Department of Veterans Affairs Medical Center, San Francisco, CA, USA

e Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, CA, USA

f Department of Psychology, University of California, Los Angeles, CA, USA

g Department of Psychology, Harvard University, Cambridge, MA, USA

h VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA

ARTICLE INFO

ABSTRACT

Article history:

Received 24 October 2013

Received in revised form 22 January 2014

Accepted 24 January 2014

Available online 12 March 2014

Background: Pervasive social cognition deficits are evident early in the course of schizophrenia and are directly linked to functional outcome, making them an important target for intervention. Here, we tested the feasibility of use, and initiated the evaluation of efficacy, of a novel, neuroplasticity-based online training program (SocialVille) in young adults with schizophrenia.

Methods: Schizophrenia patients (n = 17) completed 24 hours of online SocialVille game play either from

Keywords:

Social cognition

Early psychosis

First episode

Computerized training

Cognitive remediation

home or at a clinic, over a 6-10 week period. We examined training feasibility, gains on the SocialVille exercises relative to matched healthy controls (n = 17), and changes on measures of social cognition, social functioning, global functioning and motivation.

Results: Subjects adhered to training requirements, and rated SocialVille in the medium to high range in satisfaction, enjoyment, and ease of use. Subjects demonstrated significant, large improvements on the speeded SocialVille tasks, and small to moderate improvements on the working memory tasks. Post-training performance on the SocialVille tasks were similar to initial performance of the healthy controls. Subjects also showed improvements on standard measures of social cognition, social functioning, and motivation. No improvements were recorded for emotion recognition indices of the MSCEIT, or on quality of life scales.

Conclusion: This study provides an initial proof of concept for online social cognition training in schizophrenia. This form of training demonstrated feasibility and resulted in within-subject gains in social functioning and motivation. This pilot study represents a first step towards validating this training approach; randomized controlled trials, now underway, are designed to confirm and extend these findings.

© 2014 Elsevier Inc. All rights reserved.

1. Introduction

In recent years, special emphasis has been given to the need for early intervention in schizophrenia (Insel, 2010; McGorry, 2011; Wykes et al., 2007). It is now acknowledged that the early phase of psychotic illness is crucial in terms of the emergence of a range of cognitive deficits that have prognostic implications (Bartholomeusz and Allott, 2012; Birchwood et al., 1990) - and that early intervention can potentially prevent further worsening of symptoms and improve functioning. An important target for early intervention is the domain of social cognition, the mental operations that underlie understanding, interpretation and perception of social information (Fiske and Taylor, 1991). Severe social cognition deficits, often comparable to those seen in chronic patients, have been repeatedly documented in early-phase schizophrenia (e.g. Bertrand et al., 2007; Green et al., 2012a; Williams et al., 2008). These deficits span the domains of affect perception (e.g. Edwards et al., 2001), social cue perception (including gaze perception, e.g. Hooker and Park, 2005; Rosse et al., 1994; Tso et al., 2013; Tso et al., 2012), theory of mind (ToM; e.g. Bertrand et al., 2007) and attributional style (e.g. Humphreys and Barrowclough, 2006). Importantly, social cognition deficits have been strongly associated with poor functional outcome in schizophrenia (e.g. Fett et al., 2010). Specifically, affect recognition and social perception have been each linked with community functioning, social problem solving and social skills (Couture et al., 2006; Irani et al., 2012); ToM, as well as affect perception and social perception, have been found to mediate the relationship between neurocognition and functional outcome (Addington et al., 2006a, b; Billeke and Aboitiz, 2013; Brekke et al., 2005; Couture et al., 2011; Fett et al., 2010; Sergi et al., 2006).

Surprisingly, however, only a few studies to date have examined the direct effects of social cognition training in young adult or early psychosis patients (Bartholomeusz et al., 2013; Eack et al., 2007; Eack et al., 2009), and none have evaluated a computerized intervention. Studies testing the effects of Cognitive Enhancement Therapy, a computer-based cognitive training with group-based social skills training (Eack et al., 2007,2011; Eack et al., 2009) and of SC1T, a social cognitive group intervention (Combs et al., 2007; Penn et al., 2005) in first episode patients report promising effects on neurocognitive, social cognitive and functional outcome measures. However, these encouraging outcomes are limited by the practicality of applying these treatments in many clinical settings, given long treatment durations, the need for a trained clinician team, and the necessity of organizing patient groups for program delivery.

Computer-based training has the potential to overcome these limitations by allowing individualized treatment that can be more flexibly performed from home, and at a considerable cost savings (Ventura et al., 2013). A few recent studies (Frommann et al., 2003; Hooker et al., 2012; Hooker et al., 2013; Russell et al., 2006; Sacks et al., 2013; Wolwer et al., 2005) tested computerized training of facial affect recognition and mental state decoding in chronic patients, with promising results in emotion perception, management and social functioning (Kurtz and Richardson, 2012). These results in chronic patients show that improving social cognition and social skills using a single-user, computer-based intervention is feasible and is potentially beneficial. However, it is unclear whether younger patients would comply with computerized social cognitive training, and with the requirement to individually train from home for several hours each week over a several week long therapeutic epoch.

Here, we examined the feasibility and initial efficacy of a new, online training program (SocialVille by Brain Plasticity Institute of Posit Science; Nahum et al., 2013a), in young adults with schizophrenia. SocialVille aims to treat social cognition deficits using the principles of neuroplasticity-based learning (Merzenich, 2013; Nahum et al., 2013b), by targeting the impaired brain systems underlying social cognition rather than the impaired social behaviors per-se that are targeted by molar social skills training approaches. The SocialVille exercises aim to improve efficiency of stimulus representation and processing speed in the specific neural systems that underlie social cognition, and have been shown to function abnormally in schizophrenia (e.g. Crossley et al., 2009). The rationale behind this approach was successfully applied to address general cognitive deficits in chronic schizophrenia (e.g. Fisher et al., 2009; Subramaniam et al., 2012): that directly strengthening the fidelity of representations of sociallyrelevant information in the brain should improve an individuals social behavior. The SocialVille exercises employ psychophysical principles of training and exploit implicit learning mechanisms: they continuously adapt in difficulty, based on individual performance. The user learns through tasks that involve many socially-relevant stimulus examples while given feedback on correct and incorrect discriminations. These neuroplasticity-based principles, which provided the basis for the construction of SocialVille, have been recently summarized in a review paper (see Nahum et al., 2013b for full details) (Fig. 1).

To assess feasibility, we predicted that at least 75% of enrolled participants would complete the training and that they would report satisfaction with the exercises with ratings of over 4 on the 7-point Likert scale items. We measured learning rates by assessing improvement on the SocialVille exercises. We hypothesized that participants would show improvements on the trained exercises, as well as learning generalization to proximal social cognition measures. We explored whether more distal measures of functioning and motivation would improve with this form of computerized training.

2. Materials and methods

2.1. Participants

Schizophrenia (SZ) subjects (n = 17) and matched healthy controls (HC) (n = 17) completed the study protocol in two sites at university-based early psychosis programs at the University of California, San Francisco and Los Angeles (Table 1). SZ subjects met the following inclusion/exclusion criteria: Diagnosis of schizophrenia, schizophreniform, or schizoaffective disorder via the Structured Clinical Interview for DSM-IV (SCID-IV; First et al., 1996); good general physical health; age 18-31 years; Premorbid IQ > 70 (estimated using WTAR; Wechsler, 2001); No neurological disorder or history of traumatic brain injury; No substance dependence or serious substance use in the past 6 months. SZ Symptom severity at baseline was assessed using the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987). We note that nine subjects were within 2 years of psychosis onset, seven more within 5 years, and that only one was more than five years from onset (7.2 years). All subjects had achieved clinical stability (outpatient status for at least 3 months prior to study participation) and were stable on psychiatric medications for at least one month prior to study onset.

UCSF site participants (n = 10) were either active or former patients at the early psychosis clinic (EPC), which provides medication management and some psychotherapy; for the duration of the study, 4 were receiving no treatment, 3 were receiving outside treatment and 3 were seen at the EPC. UCLA participants were enrolled in the Aftercare Program protocol, and all were receiving psychosocial treatment, medication, and case management there. None of the UCLA participants were in other cognitive remediation training during their SocialVille training.

HC subjects (status verified through the SCID-Nonpatient edition) were recruited to match the SZ cohort at a group level in terms of age, gender and education (Table 1).

2.2. Study Procedures

SZ subjects: Following informed consent, eligible participants underwent baseline assessments. Subjects from the UCSF clinic (n = 10) were then loaned laptop computers, given logins and asked to complete the intervention from home. The UCLA clinic

Table 1

Sample characteristics of the patient and healthy control groups.

Variable

Schizophrenia (SZ)(n = 17)

Healthy control (HC)(n = 17)

Analysesa

X2

df

P

Male/female

13/4

12/5

.151

1

.69

Mean (SD)

Mean (SD)

F

df

P

Age, years

23.8 (±3.2)

23.6 (±3.6)

.20

32

.84

Education, years

12.8 (±1.7)

13.5 (±1.5)

-1.28

32

.21

IQ (WTAR)

101.2 (±16.2)

110.7 (±11.3)

-1.95

32

.06

Time since psychosis onset, months

PANSSb

38.5 (±23.9)

-

-

-

-

Positive

13.1 (±1.2)

-

-

-

-

Negative

16.2 (±1.4)

-

-

-

-

General psychopathology

27.8 (±2.1)

-

-

-

-

TOTAL

57.1 (±4.1)

-

-

-

-

Medication (chlorpromazine equivalent)

281.5 (±31.1)

N/A

-

-

-

Total training times are marked in bold type.

a 1ndependent samples t-test or chi-square analysis for categorical data. b Positive and Negative Syndrome Scale.

subjects (n = 7) came into the clinic twice a week to complete the intervention there, as they were already coming to the clinic twice each week anyway. Subjects from both sites were asked to complete 24 hours of training with the SocialVille online program (1-2 hours per day, 2-5 days per week for 6-12 weeks). Following training completion, subjects completed the post-training assessment battery.

HC subjects: following consent and initial screening, participants completed the SocialVille exercise-based assessments only, i.e. one block of every SocialVille social cognitive exercise (see details below). Subjects received monetary compensation for participating in the study.

Assessments were conducted by psychology practicum graduate students (UCSF site) or by case managers and research assistants (UCLA site); assessors were not directly paid by Posit Science for their services.

2.3. Social Cognition Training Intervention (SocialVille)

SocialVille is an online program designed to treat social cognition deficits in schizophrenia. It consists of 19 computerized exercises targeting speed and accuracy of neural functions dedicated to processing of social information (e.g. Nahum et al., 2013a; Nahum et al., 2013b). Specifically, the SocialVille exercises target the social cognitive domains of affect perception (both visual and vocal), social cue perception, ToM and self-referential processing (see Table 2). A full description of the SocialVille program is provided in Supplemental Document 1. Each SocialVille session consisted of 6 exercise blocks, each taking about 10 minutes to complete.

2.4. Outcome measures

2.4.1. SocialVille training program feasibility and ease of use:

2.4.2. Social Cognition Learning: SocialVille Exercise-based Assessments

Performance levels on the SocialVille exercises (i.e. social cognition abilities) were assessed at baseline (SZ and HC participants) and post-training (SZ only) using the exercise-based assessments embedded in the SocialVille program. Each assessment is a single blockof each exercise (except for two exercises, see Table 2), for which performance threshold is determined using an adaptive staircase algorithm (e.g. Levitt, 1971) within 5-15 minutes. A full description of the exercise-based assessments is included in Supplemental Document 1.

For each exercise-based assessment, we derived normalized (Z) scores relative to the mean score of the HC group on that assessment. This was done to allow the creation of composite scores by summing assessment scores across domains, and to more easily visually depict test scores across a variety of scales (e.g. reaction time in ms, number of correct items, % correct, etc.).

The remainder of the outcome measures were administered to the SZ group only:

2.4.3. Social cognition outcome measures

2.4.4. Functioning

Table 2

SocialVille exercises and total time spent on each exercise during 24 hour course of training.

Exercise name

Exercise description

Time (hours)

Speeded exercises

S1 Gaze ID*

A speeded gaze identification task (select the peripheral object)

1

S Gaze Match*

A speeded gaze matching task (match gaze direction of target face)

1

S Face Match*

A speeded face matching task

1.2

S Face Emotion ID*

A speeded facial emotion identification task (stills)

2

S Face Emotion ID (Clips)*

A speeded facial emotion identification task (video clips)

1.2

S Face Emotion Match

A speeded facial emotion matching task (stills)

1.6

S Valence Match*

Delayed match of the valence of a target picture

1

Facial Emotion CPT3*

Withhold response for neutral faces, press for emotional faces

1

Emotion Maintenance*

Select the smiling face when faces keep changing

1

Facial Affect ToM*

Select the correct affective (facial) response in a given situation

1.2

Total time on speeded exercises

WM2 exercises

12.2

WM Facial Emotions

Match pairs of facial emotion cards in a WM task

1

WM Facial Emotion (clips)*

Match pairs of facial emotions and labels in a WM task

1.5

WM Vocal Emotions

Match pairs of vocal emotion cards in a WM task

1

Faces Span

Arrange the faces in the order they were presented

1.3

Facial Emotion Span

Arrange the facial emotions in the order they were presented

1

Face Stories Span*

Memorize sequences of faces and personal facts on the faces

1

Total time on WM exercises

Other exercises

6.8

Vocal Emotion ID

Identify the prosody of the sentence with the neutral content

1.2

Social StoriesNM

Answer questions regarding social interactions in a segmented story

2.5

Vocal Affect ToMNM

Select the correct prosody

(vocal affect)

response in a given situation

1.5

Total time on other exercises

5.2

Total training times are marked in bold type.

* A task for which there was a significant improvement in threshold following training; NMA task for which threshold performance was not measured.


2.4.5. Self-report motivation measures

2.5. Data analysis

All variables were screened and normally distributed after winsorising of outlying values. Group differences in gender were tested using a chi-square test. Independent Samples t-tests tested for group differences in age, education level, IQ and the exercise-based assessments.

Paired samples two-tailed t-tests were used to examine whether SZ participants made significant gains on the training exercises and to test changes from baseline to post-training on all other outcome measures. Two-tailed tests with no correction for multiple comparisons were used because gains were hypothesized and because of the preliminary nature of the study.

3. Results

3.1. Clinical and demographic data

Demographic and clinical data is summarized in Table 1. The HC and SZ groups did not differ significantly in gender or education. As is typical with this population, the groups differed slightly in IQ, with the difference approaching statistical significance (F(1,32) = 1.95; p = .06).

3.2. SocialVille program feasibility

3.2.1. Attrition Rate

Twenty-two (22) SZ participants were recruited (UCSF: n = 13; UCLA: n = 9). Seventeen completed the study. Five subjects (3 from UCSF and 2 from UCLA) dropped out at initial phases (4 before being given the training laptop and one after completing 2 training sessions). The reasons provided for dropping out were as follows: due to increased stress at work/school (n = 2), hospitalization (n = 1), boredom (n = 1) and unknown reason (n = 1). Attrition rate was not different for the two sites (23% and 22% at UCSF and UCLA, respectively).

3.2.2. Compliance with online SocialVille training

Adherence to the online training schedule. Subjects were asked to complete 2-5 one-hour sessions of training per week for 6-12 weeks (24 sessions total). On average, subjects took 8.1 ± 2.5 weeks to complete training (range: 5-12 weeks), with only 4 subjects


taking more than 10 weeks to complete the program. The average number of weekly sessions completed was 3.15 ± 0.8.

SocialVille program rating. Following training, subjects rated their satisfaction in playing SocialVille on a 7-point Likert scale. The following averaged ratings were obtained: (a) Satisfaction rating (I felt satisfied after the training') of 5 ± 1.9 (5 corresponds to somewhat agree'); (b) Program clarity rating (the exercise instructions and tutorials were easy to understand') of 6 ± 1 (6 corresponds to mostly agree'); (c) Ease of navigation rating (the program was easy to navigate') of 5.8 ± 1.3; (d) Ease of use rating (the program makes it quick and easy for me to start playing each day') of 4.7 ± 2 and ease of fit into daily schedule (the program easily fits into my daily schedule') of 4.5 ± 1.8; (e) Attractiveness rating (the program graphics were attractive') of 4.5 ± 1.9; (f) Program usage difficulty (the program was difficult to use) was rated as 3.06 ± 2.1 (somewhat disagree'); (g) Security concerns of using the online program (I was worried about the security of my log-in account') were

3.3. Social Cognition Learning: SocialVille Exercise-based Assessments

For 17 of the 19 SocialVille exercises, we calculated a normalized z-score for the SZ group relative to the HC baseline performance on each exercise. For two exercises, no thresholds were derived due to a data recording error, hence they were not included in the normalized composites (see Table 2). Results are summarized in Fig. 2, in Supplementary Figure 1 and in Supplementary Table 1. On the speededSocialVille exercises (i.e. tasks that required speeded processing of stimulus on every trial, Fig. 2A), significant z-score changes from baseline to post-training were evident on all but one of the exercises (two were at trend-level). Following training, patients' performance on these tasks was similar to that of the initial performance of the HC group (see Supplementary Figure 1). On the 6 WM-based SocialVille exercises, improvements were evident on 5 of the exercises, however only one of these reached statistical significance (Figure 2B). We further derived composite scores for the speeded tasks, the WM tasks and a total composite score. Significant z-score changes were seen on all composite scores (speeded: t(16) = 6.9, p < .0001; WM: t(16) = 2.6, p < .02; Total: t(16) = 7.4, p < .0001). The pre-to post-test gain on the speeded task composite score was large (1.15 SD), while the gain in the WM composite score was small to moderate (0.45 SD).

3.4. Social cognition outcome measures

Outcome measures data for the study are summarized in Table 3.

3.5. Functioning

Subjects showed a significant increase in GFS's social functioning, and no significant change in role functioning (see Fig. 4A). On the SFS there was a trend-level increase on the interpersonal communication

Table 3

Scores on the study outcome measures before (baseline) and after (post-training) 24 hours of SocialVille training.

Outcome measure

Baseline

Post-training

Stats

Mean (± SD)

Mean (± SD)

t

p

Effect size

Emotional Prosody ID (PROID)

# Correct responses

40.06 (±1.87)

42.9 (±1.96)

1.8

0.09

0.37

Median RT

3940 (±129)

3540 (±144)

2.94

< .01

0.71

Facial Memory (Penn Test) - immediate recall

# Correct responses

30.06 (±1.1)

31.6 (±1.1)

1.4

0.16

0.22

Median RT

2025 (±164)

1679 (±116)

3.14

< .007

0.6

Facial Memory (Penn Test) - delayed recall

# Correct responses

30.5 (±1.1)

31.1 (±1.4)

.49

0.63

0.11

Median RT

1870 (±217)

1415 (± 87)

2.43

< 0.03

0.73

MSCEIT

Perceiving Emotions SS1

107.3 (±3.3)

105.1 (±4.1)

.63

0.54

0.12

Managing Emotions SS

91.9 (±3.7)

94.1 (±4.9)

.72

0.42

0.15

Global Functioning Scale (GFS)

GFS: Role Current SS

5.2 (±0.7)

5.5 (±0.6)

.85

0.41

0.085

GFS: Social Current SS

5.8 (±0.47)

6.6 (±0.43)

2.5

< 0.03

0.4

Social Functioning Scale (SFS)

SFS: Interpersonal Communication SS

113.2 (±3.9)

119.1 (±4.6)

1.6

0.12

0.37

Behavioral Inhibition/Behavioral Activation Scale (BIS/BAS)

BIS Total

3.11 (±0.12)

2.96 (±0.14)

2.2

0.04

0.28

BAS Reward Responsiveness

3.24 (±0.14)

3.14 (±0.18)

1.07

0.39

0.17

BAS Drive

2.52 (±0.18)

2.71 (±0.2)

1.76

0.09

0.24

BAS Fun Seeking

2.94 (±0.13)

2.99 (±0.18)

.70

1

0

The Temporal Experience of Pleasure Scale (TEPS)

TEPS: Anticipatory SS

38.06 (±2.6)

41.1 (±2.5)

2.29

< 0.04

0.28

TEPS: Consummatory SS

33 (±2.2)

32.3 (±2.2)

.54

0.6

0.07

Quality of Life Scale Total

3.29 (±0.3)

3.26 (±0.3)

.17

0.86

0.02

Significant or trend-level p values are marked in bold type. 1 SS: Subscale.

subscale (Fig. 4B), and no significant change on the other SFS subscales. There were no significant changes on the Quality of Life Scale.

3.6. Motivation/reward sensitivity

4. Discussion

We tested the feasibility and preliminary efficacy of SocialVille, a new online training program targeting deficits in social cognitive processing speed and working memory (WM) in young adults with schizophrenia. Following 24 hours of training performed over a 6-12 week period, we found: relatively high adherence with the training requirements, satisfaction with the exercises; significant pre- to posttraining improvements on the SocialVille social cognition exercise composite scores; significant improvements on proximal measures of social cognition (prosody identification and facial memory); and significant improvements on social functioning, motivation and reward sensitivity.

These findings suggest that an online training approach is feasible in early psychosis, and that social cognitive deficits may be addressed early in the course of schizophrenia. Given the importance of early intervention (Birchwood et al., 1998; Marshall and Rathbone, 2011), and the link between social cognition and functional outcome in schizophrenia (Fett et al., 2010; Sergi et al., 2007) this study provides an initial scalable way of targeting a set of critical core deficits. Although promising, the results are preliminary and not case-controlled. An appropriately powered, randomized controlled trial is required to determine whether these effects are replicable.

To the best of our knowledge, our study is the first to demonstrate the feasibility of online social cognitive training in young adults with



schizophrenia (see recent review in Bartholomeusz and Allott, 2012). Two other studies tested the feasibility of social cognitive training in early psychosis, both using group-based interventions administered in clinic (SCIT, Bartholomeusz et al., 2013; CET, Eack et al., 2007). CET uses computerized coldcognition training with group-based social skills training. SCIT is a group-based therapy as well, comprised of 18 sessions focused on three phases of understanding emotions, social cognitive biases, and integration, in which trainees practice the acquired social skills in everyday situations (Penn et al., 2005). These two interventions, although including some computer-assisted parts, are administered in a therapist-instructed group format. Here, we show the feasibility of a fully-computerized intervention, which can be completed either at home or at the clinic, using internet-connected computers. Gains were obtained after a relatively short period of treatment (less than 12 weeks). The attrition rate in our study is similar to other training studies in young clinical populations (e.g., Marshall and Rathbone, 2011). Moreover, the high compliance with the training schedule indicates that this form of individual online training is feasible with young adults, comparable to clinic-based group training. We note that at-home participants needed weekly phone-calls and/or emails to reliably adhere to training requirements, thus clinical staff involvement in the form of solution-focused conversations was still required. Future studies will determine whether combining our computer-based training with psychosocial interventions (e.g. social skills groups) enhances generalization. Still, our study provides the first demonstration of a potentially highly-scalable form of treatment that could be easily used in practically every household or clinical facility equipped with an internet connection, thus providing treatment options to under-resourced areas and to patients who are unable or unwilling to come in to the clinic.

Study participants made larger gains on the speededSocialVille composite (i.e. processing speed of social information), whereas improvements on the WMcomposite (i.e. WM manipulations of social information) were smaller. The fact that large pre-post gains were found in processing speed is encouraging: recent cold cognition’ studies consider speed of processing deficits among the largest cognitive impairments in recent-onset schizophrenia (Mesholam-Gately et al., 2009; Milev et al., 2005) and hence an important target for early intervention. While speed of processing during cold cognition” tasks and social cognitive tasks may be quite different, our results suggest that speed-of-processing deficits of social information are evident in early psychosis, and that training drives improvements in this fundamental ability.

Although there are currently no rigorous studies of computerized social cognition training in early schizophrenia, our outcomes are comparable to recent reports of social cognition training in chronic schizophrenia (see Kurtz and Richardson, 2012 for review; Lindenmayer et al., 2013), and further strengthen the notion that social cognition is linked to social functioning and functional outcome in schizophrenia (Billeke and Aboitiz, 2013; Brekke et al., 2005). Interestingly, while SocialVille includes training on facial affect, we did not find significant changes on the MSCEIT emotion perception and management subscales. However, our findings still suggest that subjects improved on emotion recognition abilities, as is evident by improvements on SocialVille emotion exercises and on vocal affect recognition. We hypothesize that this negative result is likely due to the focus of SocialVille on processing speed, while affect perception and management is measured differently in the MSCEIT (see similar arguments in Roberts et al., 2006). The short duration of training relative to that included in other studies (e.g. Eack et al., 2007; Eack et al., 2009) might have also contributed to the lack of improvement on the MSCEIT.

Following training, participants improved on motivation and reward sensitivity: participants showed decreased behavioral inhibition and increased drive, as well as increased anticipatory pleasure. To our knowledge, this is the first demonstration of changes in motivation following cognitive training in early schizophrenia. Interestingly, motivation is generally considered a stable trait in healthy individuals, not subject to change. A few recent reports (e.g. Gard et al., 2009; Green et al., 2012b) have shown that motivation plays a significant and mediating role between neurocognition, social cognition and functional outcome. Our preliminary finding that motivation can be enhanced with social cognitive training provides strong support for this model, and further stresses the importance of targeting social cognition in schizophrenia (see also Choi and Medalia, 2010).

Our study had several limitations. These include the small sample size, the lack of a control group, and the fact that participants were provided remuneration for participation in the study. Further, we cannot rule out practice effects or non-specific effects of study participation. These factors all limit our ability to attribute improvements to the SocialVille training itself. Furthermore, since the main goal of the study was to establish feasibility in early schizophrenia patients, no general cognition and symptom outcome measures were included. Future, well-controlled studies are required to further establish the efficacy of computerized social cognitive training administered with no additional neurocognitive training (see discussion in Pinkham and Harvey, 2013). Also, the results of our pilot study do not rule out the possibility that improvements are driven by non-specific effects of training such as increased attention span or general improved processing speed. Still, results from several recent studies (e.g. Fisher et al., 2009; Lindenmayer et al., 2013; Sacks et al., 2013) imply that non-social cognitive training does not improve social cognition and social function. Finally, we note that there is currently no consensus on the best social cognition outcome measures to be used in intervention studies, as many of them are considered to have poor psychometric characteristics (see Pinkham et al., 2013). Future studies should consider applying additional or different outcome measures, given new psychometric information on outcome measures (see, for example, Green et al., 2013; Kern et al., 2013).

We conclude that SocialVille is a promising intervention which is feasible and resulted in initial positive outcomes in social cognition, social functioning, and motivation in young individuals with schizophrenia. Given the importance of early intervention, and the lack of effective treatment options, there is a clear need for effective, scalable treatments. Future randomized controlled trials will determine whether these preliminary findings are replicable and are needed to discover the active ingredientsof training that allow for learning to transfer to everyday functioning.

Supplementary data to this article can be found online at http:// dx.doi.org/10.1016/j.scog.2014.01.003.

Role of Funding Source

The study was funded by a National Institute of Mental Health SBIR Grant (1R43MH091793-01A1) to author M.N.

Contributors

Author M.N. developed the training program, designed the study, supervised the study, conducted data analysis and wrote the first draft of the manuscript. Author M.F. helped supervise the UCSF site study, conducted data analyses and wrote the manuscript. Author R.L. supervised the trial and managed patient recruitment and participation. Author G.P. supervised the assessments and outcome measures. Authors J.V. and K.N. supervised the UCLA site study and provided input on the development of the training program. Authors

Conflict of interest

The social cognitive training software used in this study (SocialVille) was developed by Posit Science. Dr. Nahum is a paid employee of Posit Science and was the main developer of the program. In addition, she received SBIR grant from NIMH to develop and test the SocialVille software. Dr. Merzenich is the founder and CSO of Posit Science. Drs. Vinogradov, Hooker, Green and Ventura are paid consultants to Posit Science and were all involved in the construction of the training program. Dr. Nuechterlein is an unpaid consultant to Posit Science, and holds research grants from Janssen Scientific Affairs and Genentech. He serves as a consultant to Otsuka and Genentech. Dr. Vinogradov serves on advisory boards for Genentech, Envivo, and Hoffman-LaRoche. Dr. Green reports having been a consultant to Abbott Laboratories (AbbVie), Biogen, DSP and Roche. He is a member of the scientific board for Mnemosyne, and has received research funds from Amgen. Dr. Ventura has received research support from Brain Plasticity Inc. (a company merged with Posit Science) and from Janssen Scientific Affairs. Dr. Loewy has received research funding from Genentech.

Drs. Fisher and Poelke report no conflicts of interest.

References

Addington, J., Saeedi, H., Addington, D., 2006a. Facial affect recognition: A mediator between cognitive and social functioning in psychosis? Schizophr. Res. 85 (1-3), 142-150.

Addington, J., Saeedi, H., Addington, D., 2006b. Influence of social perception and social knowledge on cognitive and social functioning in early psychosis. Br. J. Psychiatry: J. Mental. Sci. 189, 373-378.

Bartholomeusz, C.F., Allott, K., 2012. Neurocognitive and social cognitive approaches for improving functional outcome in early psychosis: Theoretical considerations and current state of evidence. Schizophr. Res. Treat. 2012, 815315.

Bartholomeusz, C.F., Allott, K., Killackey, E., Liu, P., Wood, S.J., Thompson, A., 2013. Social cognition training as an intervention for improving functional outcome in first-episode psychosis: A feasibility study. Early Interv. Psychiatry 7 (4), 421-426.

Bertrand, M.C., Sutton, H., Achim, A.M., Malla, A.K., Lepage, M., 2007. Social cognitive impairments in first episode psychosis. Schizophr. Res. 95 (1-3), 124-133.

Bilker, W.B., Brensinger, C., Kurtz, M.M., Kohler, C., Gur, R.C., Siegel, S.J., Gur, R.E., 2003. Development of an abbreviated schizophrenia quality of life scale using a new method. Neuropsychopharmacology 28 (4), 773-777.

Billeke, P., Aboitiz, F., 2013. Social cognition in schizophrenia: From social stimuli processing to social engagement. Front. Psychiatry 4, 4.

Birchwood, M., Smith, J., Cochrane, R., Wetton, S., Copestake, S., 1990. The Social Functioning Scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br. J. Psychiatry 157, 853-859.

Birchwood, M., Todd, P., Jackson, C., 1998. Early intervention in psychosis. The critical period hypothesis. Br. J. Psychiatry Suppl. 172 (33), 53-59.

Brekke, J., Kay, D.D., Lee, K.S., Green, M.F., 2005. Biosocial pathways to functional outcome in schizophrenia. Schizophr. Res. 80 (2-3), 213-225.

Carver, C.S., White, T.L., 1994. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales. J. Pers. Soc. Psychol. 67, 1126-1133.

Choi, J., Medalia, A., 2010. Intrinsic motivation and learning in a schizophrenia spectrum sample. Schizophr. Res. 118 (1-3), 12-19.

Combs, D.R., Adams, S.D., Penn, D.L., Roberts, D., Tiegreen, J., Stem, P., 2007. Social Cognition and Interaction Training (SCIT) for inpatients with schizophrenia spectrum disorders: Preliminary findings. Schizophr. Res. 91 (1-3), 112-116.

Cornblatt, B.A., Auther, A.M., Niendam, T., Smith, C.W., Zinberg, J., Bearden, C.E., Cannon, T.D., 2007. Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia. Schizophr. Bull. 33 (3), 688-702.

Couture, S.M., Granholm, E.L., Fish, S.C., 2011. A path model investigation of neurocognition, theory of mind, social competence, negative symptoms and real-world functioning in schizophrenia. Schizophr. Res. 125 (2-3), 152-160.

Couture, S.M., Penn, D.L., Roberts, D.L., 2006. The functional significance of social cognition in schizophrenia: A review. Schizophr. Bull. 32 (Suppl 1), S44-S63.

Crossley, N.A., Mechelli, A., Fusar-Poli, P., Broome, M.R., Matthiasson, P., Johns, L.C., Bramon, E., Valmaggia, L., Williams, S.C., McGuire, P.K., 2009. Superior temporal lobe dysfunction and frontotemporal dysconnectivity in subjects at risk of psychosis and in first-episode psychosis. Hum. Brain Mapp. 30 (12), 4129-4137.

Eack, S.M., Greenwald, D.P., Hogarty, S.S., Cooley, S.J., DiBarry, A.L., Montrose, D.M., Keshavan, M.S., 2009. Cognitive enhancement therapy for early-course schizophrenia: Effects of a two-year randomized controlled trial. Psychiatr. Serv. 60 (11), 1468-1476.

Eack, S.M., Hogarty, G.E., Greenwald, D.P., Hogarty, S.S., Keshavan, M.S., 2007. Cognitive enhancement therapy improves emotional intelligence in early course schizophrenia: Preliminary effects. Schizophr. Res. 89 (1-3), 308-311.

Eack, S.M., Hogarty, G.E., Greenwald, D.P., Hogarty, S.S., Keshavan, M.S., 2011. Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: Results from a two-year randomized trial. Res. Soc. Work Pract. 21 (1), 32-42.

Edwards,J., Pattison, P.E.,Jackson, H.J., Wales, R.J., 2001.Facial affect and affective prosody recognition in first-episode schizophrenia. Schizophr. Res. 48 (2-3), 235-253.

Fett, A.K., Viechbauer, W., et al., 2010. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: A meta-analysis. Neurosci. Biobehav. Rev. 35 (3), 573-588.

First, M.B., Spitzer, R.L., Gibbon, M., Williams,J.B.W., 1996. Structured clinical interview for DSM-IV axis I disorders, Clinician Version (SCID-CV). American Psychiatric Association, Washington, DC.

Fisher, M., Holland, C., Merzenich, M.M., Vinogradov, S., 2009. Using neuroplasticitybased auditory training to improve verbal memory in schizophrenia. Am.

J. Psychiatry 166 (7), 805-811.

Fiske, S.T., Taylor, S.E., 1991. Social cognition, 2nd ed. McGraw Hill, New York, NY.

Frommann, N., Streit, M., Wolwer, W., 2003. Remediation of facial affect recognition impairments in patients with schizophrenia: A new training program. Psychiatry Res.117 (3), 281-284.

Gard, D.E., et al., 2006. Anticipatory and consummatory components of the experience of pleasure: A scale development study. J. Res. Pers. 40, 1086-1102.

Gard, D.E., Kring, A.M., Gard, M.G., Horan, W.P., Green, M.F., 2007 Jull. Anhedonia in schizophrenia: distinctions between anticipatory and consummatory pleasure. Schizophr. Res. 93 (1-3), 253-260.

Gard, D.E., Fisher, M., Garrett, C., Genevsky, A., Vinogradov, S., 2009. Motivation and its relationship to neurocognition, social cognition, and functional outcome in schizophrenia. Schizophr. Res. 115 (1), 74-81.

Green, M.F., Bearden, C.E., Cannon, T.D., Fiske, A.P., Hellemann, G.S., Horan, W.P., Kee,

K. , Kern, R.S., Lee, J., Sergi, M.J., Subotnik, K.L., Sugar, C.A., Ventura, J., Yee, C.M., Nuechterlein, K.H., 2012a. Social cognition in schizophrenia, Part 1: Performance across phase of illness. Schizophr. Bull. 38 (4), 854-864.

Green, M.F., Hellemann, G., Horan, W.P., Lee, J., Wynn, J.K., 2012b. From perception to functional outcome in schizophrenia: Modeling the role of ability and motivation. Arch. Gen. Psychiatry 69, 1216-1224.

Green, M.F., Lee, J., Ochsner, K.N., 2013. Adapting social neuroscience measures for schizophrenia clinical trials, part 1: Ferrying paradigms across perilous waters. Schizophr. Bull. 39 (6), 1192-1200.

Gur, R.C., Ragland, J.D., Moberg, P.J., Bilker, W.B., Kohler, C., Siegel, S.J., Gur, R.E., 2001. Computerized neurocognitive scanning: II The profile of schizophrenia. Neuropsychopharmacology 25 (5), 777-788.

Hooker, C., Park, S., 2005. You must be looking at me: The nature of gaze perception in schizophrenia patients. Cogn. Neuropsychiatry 10 (5), 327-345.

Hooker, C.I., Bruce, L., Fisher, M., Verosky, S.C., Miyakawa, A., Vinogradov, S., 2012 Augg. Neural activity during emotion recognition after combined cognitive plus social cognitive training in schizophrenia. Schizophr. Res. 139 (1-3), 53-59. http:// dx.doi.org/10.1016/j.schres.2012.05.009.

Hooker, C.I., Bruce, L., Fisher, M., Verosky, S.C., Miyakawa, A., D'Esposito, M., Vinogradov, S., 2013 Aug 30. The influence of combined cognitive plus social-cognitive training on amygdala response during face emotion recognition in schizophrenia. Psychiatry Res. 213 (2), 99-107. http://dx.doi.org/10.1016/j.pscychresns.2013.04.001.

Humphreys, L., Barrowclough, C., 2006. Attributional style, defensive functioning and persecutory delusions: Symptom-specific or general coping strategy? Br. J. Clin. Psychol. 45 (Pt 2), 231-246.

Insel, T.R., 2010. Rethinking schizophrenia. Nature 468 (7321), 187-193.

Irani, F., Seligman, S., Kamath, V., Kohler, C., Gur, R.C., 2012. A meta-analysis of emotion perception and functional outcomes in schizophrenia. Schizophr. Res. 137 (1-3), 203-211.

Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13 (2), 261-276.

Kern, R.S., Penn, D.L., Lee, J., Horan, W.P., Reise, S.P., Ochsner, K.N., Marder, S.R., Green, M.F., 2013. Adapting social neuroscience measures for schizophrenia clinical trials, part 2: Trolling the depths of psychometric properties. Schizophr. Bull. 39 (6), 1201-1210. http://dx.doi.org/10.1093/schbul/sbt127.

Kurtz, M.M., Richardson, C.L., 2012. Social cognitive training for schizophrenia: A meta-analytic investigation of controlled research. Schizophr. Bull. 38 (5), 1092-1104.

Levitt, H., 1971. Transformed up-down methods in psychoacoustics. The Journal of the Acoustical Society of America 49(2), Suppl 2:467 + .

Lindenmayer, J.P., McGurk, S.R., Khan, A., Kaushik, S., Thanju, A., Hoffman, L., Valdez, G., Wance, D., Herrmann, E., 2013. Improving social cognition in schizophrenia: A pilot intervention combining computerized social cognition training with cognitive remediation. Schizophr. Bull. 39 (3), 507-517.

Marshall, M., Rathbone, J., 2011. Early intervention for psychosis. Schizophr. Bull. 37 (6), 1111-1114.

Mayer, J.D., Salovey, P., Caruso, D.R., Sitarenios, G., 2003. Measuring emotional intelligence with the MSCEIT V2.0. Emotion 3 (1), 97-105.

McGorry, P., 2011. Transition to adulthood: the critical period for pre-emptive, diseasemodifying care for schizophrenia and related disorders. Schizophr. Bull. 37 (3), 524-530.

Merzenich, M.M., 2013. Soft-wired. Parnassus Publishing, San Francisco.

Mesholam-Gately, R.I., Giuliano, A.J., Goff, K.P., Faraone, S.V., Seidman, L.J., 2009. Neurocognition in first-episode schizophrenia: a meta-analytic review. Neuropsychology 23 (3), 315-336.

Milev, P., Ho, B.C., Arndt, S., Andreasen, N.C., 2005. Predictive values of neurocognition and negative symptoms on functional outcome in schizophrenia: A longitudinal first-episode study with 7-year follow-up. Am. J. Psychiatry 162 (3), 495-506.

Nahum, M., Garrett, C., Powell, B., Poelke, G., Fisher, M., al., e., 2013a. Testing the Feasibility of a Novel Computerized Neuro-plasticity based training program to remediate Social Cognition Deficits in Schizophrenia (SocialVille'), International Congress on Schizophrenia Research (ICOSR).

Nahum, M., Lee, H., Merzenich, M.M., 2013a. Principles of neuroplasticity-based rehabilitation. Prog. Brain Res. 207, 141-171.

Penn, D., Roberts, D.L., Munt, E.D., Silverstein, E., Jones, N., Sheitman, B., 2005. A pilot study of social cognition and interaction training (SCIT) for schizophrenia. Schizophr. Res. 80 (2-3), 357-359.

Pinkham, A.E., Harvey, P.D., 2013. Future directions for social cognitive interventions in schizophrenia. Schizophr. Bull. 39 (3), 499-500.

Pinkham, A.E., Penn, D.L., Green, M.F., Buck, B., Healey, K., Harvey, P.D., 2013. The Social Cognition Psychometric Evaluation Study: Results of the Expert Survey and RAND Panel. Schizophrenia bulletin.

Roberts, R.D., Schulze, R., O'Brien, K., MacCann, C., Reid, J., Maul, A., 2006. Exploring the validity of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) with established emotions measures. Emotion 6 (4), 663-669.

Rosse, R.B., Kendrick, K., Wyatt, R.J., Isaac, A., Deutsch, S.I., 1994. Gaze discrimination in patients with schizophrenia: Preliminary report. Am. J. Psychiatry 151 (6), 919-921.

Russ, J., 2008. Development of PROID, a computerized emotional prosody identification task. PennSci. J. 6 (2).

Russell, T.A., Chu, E., Phillips, M.L., 2006. A pilot study to investigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool. Br. J. Clin. Psychol. 45 (Pt 4), 579-583.

Sacks, S., Fisher, M., Garrett, C., Alexander, P., Holland, C., Rose, D., Hooker, C., Vinogradov, S., 2013. Combining computerized social cognitive training with neuroplasticity-based auditory training in schizophrenia. Clin. Schizophr. Relat. Psychoses 7 (2), 78A-86A.

Sergi, M.J., Rassovsky, Y., Nuechterlein, K.H., Green, M.F., 2006. Social perception as a mediator of the influence of early visual processing on functional status in schizophrenia. Am. J. Psychiatry 163 (3), 448-454.

Sergi, M.J., Rassovsky, Y., Widmark, C., Reist, C., Erhart, S., Braff, D.L., Marder, S.R., Green, M.F., 2007. Social cognition in schizophrenia: relationships with neurocognition and negative symptoms. Schizophr. Res. 90 (1-3), 316-324.

Subramaniam, K., Luks, T.L., Fisher, M., Simpson, G.V., Nagarajan, S., Vinogradov, S., 2012. Computerized cognitive training restores neural activity within the reality monitoring network in schizophrenia. Neuron 73 (4), 842-853.

Tso, I.F., Carp, J., Taylor, S.F., Deldin, P.J., 2013. Role of visual integration in gaze perception and emotional intelligence in schizophrenia. Schizophrenia bulletin.

Tso, I.F., Mui, M.L., Taylor, S.F., Deldin, P.J., 2012. Eye-contact perception in schizophrenia: relationship with symptoms and socioemotional functioning. J. Abnorm. Psychol. 121 (3), 616-627.

Ventura, J., Wilson, S.A., Wood, R.C., Hellemann, G.S., 2013. Cognitive training at home in schizophrenia is feasible. Schizophr. Res. 143 (2-3), 397-398.

Wechsler, D., 2001. Wechsler Test of Adult Reading: WTAR. Psychological Corporation. Williams, L.M., Whitford, T.J., Flynn, G., Wong, W., Liddell, B.J., Silverstein, S., Galletly, C.,

Harris, A.W., Gordon, E., 2008. General and social cognition in first episode schizophrenia: identification of separable factors and prediction of functional outcome using the IntegNeuro test battery. Schizophr. Res. 99 (1-3), 182-191.

Wolwer, W., Frommann, N., Halfmann, S., Piaszek, A., Streit, M., Gaebel, W., 2005. Remediation of impairments in facial affect recognition in schizophrenia: efficacy and specificity of a new training program. Schizophr. Res. 80 (2-3), 295-303.

Wykes, T., Newton, E., Landau, S., Rice, C., Thompson, N., Frangou, S., 2007. Cognitive remediation therapy (CRT) for young early onset patients with schizophrenia: an exploratory randomized controlled trial. Schizophr. Res. 94 (1-3), 221-230.

Online Social Cognition Training in Schizophrenia: A Double-Blind, Randomized, Controlled Multi-Site Clinical Trial


Mor Nahum*,1,2, Hyunkyu Lee2, Melissa Fisher3, Michael F. Green4,5, Christine I. Hooker6, Joseph Ventura5,

Joshua T. Jordan7, Annika Rose2, Sarah-Jane Kim2, Kristen M. Haut6, Michael M. Merzenich2, and Sophia Vinogradov3

1School of Occupational Therapy, Faculty of Medicine, Hebrew University, Jerusalem, Israel; 2Department of Research and Development, Posit Science Inc., San Francisco, CA; 3Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN; 4VA Greater Los Angeles, Los Angeles, CA; Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, CA; Department of Psychiatry and Behavioral Sciences, Rush University Medical Center, Chicago, IL; Department of Psychiatry, University of California, San Francisco, CA

*To whom correspondence should be addressed; School of Occupational Therapy, Faculty of Medicine, The Hebrew University, PO Box 24026, Mount Scopus, Jerusalem, 91240, Israel; tel: +972-54-732-6655, fax: +972-2-5325345, e-mail: mor.nahum@mail.huji.ac.il


Social cognition (SC), the mental operations underlying social functioning, are impaired in schizophrenia. Their direct link to functional outcome and illness status have made them an important therapeutic target. However, no effective treatment for these deficits is currently applied as a standard of care. To address this need, we have developed SocialVille—an online, plasticity-based training program that targets SC deficits in schizophrenia. Here we report the outcomes of a double-blind, controlled, randomized, multi-site clinical trial of SocialVille. Outpatients with schizophrenia were randomized to complete 40 sessions of either SocialVille (N = 55 completers) or active control (computer games; N = 53 completers) from home. The a priori co-primary outcome measures were a social cognitive composite and a functional capacity outcome (UCSD Performance-based Skills Assessment [UPSA-2]). Secondary outcomes included a virtual functional capacity measure (VRFCAT), social functioning, quality of life, and motivation. Linear mixed models revealed a group x time interaction favoring the treatment group for the social cognitive composite (b = 2.81; P < .001) but not for the UPSA-2 measure. Analysis of secondary outcome measures showed significant group x time effects favoring the treatment group on SC and social functioning, on the virtual functional capacity measure and a motivation subscale, although these latter findings were nonsignificant with FDR correction. These results provide support for the efficacy of a remote, plasticity-based social cognitive training program in improving SC and social functioning in schizophrenia. Such treatments may serve as a cost-effective adjunct to existing psychosocial treatments. Trial Registration: NCT02246426.

Key words: computerized/computer-based/treatment/e motion/SocialVille/TRuSST

Introduction

Social cognition (SC) refers to mental operations underlying social information processing.1-4 Multiple studies have shown that schizophrenia is associated with significant deficits in all core domains of SC,5-8 ranging from emotion and social cue perception8,9 through theory of mind (ToM) and empathy.7,8,10 These deficits have functional and clinical significance, as they underlie most critical factors of daily living in schizophrenia.11-13 Moreover, the degree of SC impairment is a stronger predictor of everyday function than are cognitive abilities or the severity of positive symptoms,14,15 making them an important therapeutic target.

However, there are currently no treatment methods that are broadly administered for improving SC in schizophrenia. Pharmacological treatments have only limited impact on SC in schizophrenia.16-18 Similarly, new interventions for treating general cognitive deficits in schizophrenia have also shown only modest impact on social functioning.19 Targeted SC interventions may be necessary to drive changes in SC, which may ultimately impact functioning. Most of these existing interventions are administered by trained professionals in small groups in clinics and usually focus on emotion management and social skill-building. Indeed, targeted interventions such as the Social Cognition and Interaction Training (SCIT),20,21 the Social Cognitive Skills Training (SCST)22 and the Social Cognition Enhancement Training (SCET)23 have all shown some promise in improving SC in schizophrenia, but there is limited evidence to positive changes in functional outcome (see refs.24-27).

© The Author(s) 2020. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com


Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


Unfortunately, despite their relative success, these interventions are far from becoming standard of care, potentially due to their limited scalability, stemming from the requirement for highly trained personnel and frequent clinic visits. Computerized interventions, which are rather extensively used in cognitive training, may help overcome these difficulties, as they can be delivered remotely. Still, computer-aided interventions have been rather sparsely used to target SC, potentially because there is limited face validity of computer training for social behavior, which naturally involves other people.28 The few computerized SC programs that exist are often limited in scope— targeting a subset of SC domains—and have undergone only initial testing to date.24,29,30 Furthermore, most interventions still require some mediation by professionals and have not been applied completely remotely.

Here, we test the efficacy of SocialVille, an online intervention targeting SC abilities using individualized SC exercises. SocialVille, therefore, takes a different approach than most prior social treatment programs by using principles derived from cognitive training and neuroplasticity.31,32 As such, it is designed to engage the neural systems that support SC, which, in turn, leads to improved social skills.28 For this approach, the use of a computer is a substantial advantage: the program consists of multiple training exercises targeting multiple SC abilities, ranging from lower-level affect and social cue perception, to higher level mentalizing, self-referential style, and empathy.33 In order to promote neuroplasticity, training is intensive, adaptive, and reinfor-cing31; In SocialVille, each exercise requires the user to make hundreds of discriminations of socially relevant information that gradually involve more complex, multi-modal, and ecologically valid stimuli. To date, SocialVille has shown feasibility and initial efficacy in improving SC abilities in a pilot study in schizophrenia,33 and when applied in combination with standard cognitive training in at-risk34 and chronic35 schizophrenia samples. In addition, SocialVille training was shown to improve empathy in young healthy adults.36

We conducted a double-blind, multi-site randomized controlled trial (RCT), comparing the efficacy of SocialVille training to an active control (computer games), both applied remotely from home, using internet-connected laptops. Such remote application should facilitate delivery of the intervention and increase scalability. We hypothesized that experimental group participants would show larger SC and functional capacity gains, relative to the active control group.

Methods

The full protocol of this study has been published else-where37; most relevant details of study procedures are described below.

Participants

Clinically stable adults with schizophrenia were recruited from multiple sites: San Francisco VA Medical Center (SFVAMC), University of Minnesota (UMN), University of California, Los Angeles (UCLA), Los Angeles VA Hospital (LAVA) and Rush University. Study participants met DSM-V criteria for a diagnosis of schizophrenia (assessed using the Structured Clinical Interview for DSM-IV; SCID-P38), were between 18 and 65 years of age, had an estimated IQ > 70 based on the Wechsler Test of Adult Reading (WTAR39), were clinically stable for 8 weeks prior to consent, had no more than a moderate severity rating on hallucinations and unusual thought content (a score of <4 on the Positive and Negative Syndrome Scale; PANSS40), and no active suicidal ideation with specific plan and intent (measured by the Columbia-Suicide Severity Rating Scale; C-SSRS41). Finally, participants had been maintained on a stable treatment of no more than 2 antipsychotics and/or other concomitant psychotropic treatment for at least 6 weeks prior to consent.

Study Design

Institutional review board approval was obtained at the coordinating center (WIRB Pro Number 20141695; ClinicalTrials.gov Identifier: NCT02246426) and at each trial site. All participants signed an informed consent form prior to participation in the study.

Eligible participants completed baseline assessments in the lab/clinic and were then randomly assigned to either experimental (SocialVille) or active control (casual games) training conditions (see below). Random allocation was performed using stratification by gender, education (<13 y, >13 y), and age (18-40 y, 41-65 y) to each group at each site with an allocation ratio of 1:1. To minimize the imbalance between the number of participants in each group over these factors, we employed a minimization method42 of adaptive stratification.

Participants were loaned a laptop and were asked to complete training from home for 3-5 times/wk, for a total of 40 training sessions (42 min each) over 8-12 weeks. After 16 weeks, they were asked to complete the posttraining assessments, regardless of the amount of training completed. Training coaches interacted with participants weekly by phone in order to discuss progress and provided coaching if a participant indicated difficulty in completing training. Assessment battery was repeated in the lab mid-way through training and at the completion of the entire training program (post-training assessment).

Participants were compensated for their participation in the study, receiving $200 for all screening and assessment visits, $5 for each training session completed, and a $10 bonus for every 10 training sessions completed (maximum bonus: $40).

Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


Interventions

Both interventions were deployed on an online, browser-playable platform by Posit Science. Full list of exercises is given in supplementary material 1. Participants completed 7 unique exercises/games on every training session, for about 6 minutes each, for a total of 42 minutes per session.

The SocialVille Training Program. Socialville is a computerized SC training program developed by Posit Science (see refs.33,37). It is composed of 27 unique exercises, which collectively target visual and vocal affect perception, social cue perception, ToM, self-referential style, and empathy. Training exercises are built using similar mechanisms to those of general cognitive training, but employ socially relevant stimuli, designed to improve processing speed and accuracy in the brain systems dedicated to the processing of social information.32

Active Control Training Program. We used 13 conventional, progressive, and commercially available computer games, which were shown to provide face-valid cognitive stimulation and were rated E (for everyone) by the Entertainment Software Rating Board (ESRB). Games have been embedded in the Posit Science training portal to help maintain blinding and control for potential placebo effects, as all participants underwent the same procedure. In addition, this type of control helped match expectation-based influences on performance as well as the experimental program in overall program use intensity, staff interaction, reward, and overall engagement. Importantly, these games did not have any social content or individualized progression.

Outcome Measures

Primary Outcome Measures. We used a co-primary outcome measure, composed of an SC composite, and a functional capacity measure. The a priori co-primary SC outcome measure was a composite score of 6 SC assessments, collectively assessing facial emotion recognition (The Penn Emotional Recognition Test, ER4043), prosody identification (The Prosody Identification Test, PROID44), immediate and delayed memory for faces (the Penn Faces Memory Test, PFMT),43 the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT45) managing emotions subscale and the Empathic Accuracy (EA)Task.46 The a priori functional capacity outcome measure was the UCSD Performance-based Skills Assessment (UPSA-2),47 which assesses skills in 5 areas: household chores, communication, finance, transportation, and planning recreational activities.

Secondary Outcome Measures. In addition to the primary outcomes, we assessed clinical status and symptom severity using the Positive and Negative Syndrome Scale (PANSS),40 and functioning using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT48), a virtual reality (VR) measure mimicking a real-life scenario of a shopping trip. Functioning was further assessed using the Global Functioning Scale: Social and Role (GFS49,50), Social Functioning Scale (SFS51), an abbreviated version of the Quality of Life Scale (QLS52), and the Specific Levels of Functioning Scale (SLOF53).

SC was assessed using additional measures of facial affect perception (the Morphed Faces Task),54 social perception (The Awareness of Social Inference Test, Part 3 [TASIT])55; ToM (the Faux Pas Recognition Test56,57), memory for the source of items (The Source Memory Test58), and attributional style (The Ambiguous Intentions Hostility Questionnaire, AIHQ59).

Finally, we assessed motivation, which has been found to be a critical mediator between SC and function in schizophrenia,60 using both the Temporal Experience of Pleasure Scale (TEPS61) and the Behavioral Inhibition/ Behavioral Activation Scale (BIS/BAS62), which assesses sensitivity to anticipated punishment or reward.

Data Analysis

Power calculations are reported elsewhere.37

Primary Analysis. Groups were compared on all baseline measures. Differences between groups were tested via Mann-Whitney U tests (for continuous variables) or Pearson chi-square tests (for categorical variables). Raw scores of the 6 primary SC outcomes were first converted into Z-scores, and then summed and normalized (mean of 100, SD of 15).

Since there were group differences on the primary outcomes at baseline, we implemented a propensity score framework based on demographic and primary outcomes to compare trajectories of change over time.63,64 This process effectively creates a matched sample.65

To test whether groups differed in change over time, data were analyzed according to the intent-to-treat (ITT) principle, in which all subjects randomized into either the treatment or control group were included in the analysis. This was accomplished via a linear mixed-effects model (LMM), where missing data was handled via full information maximum likelihood (FIML). FIML is a “gold standard” approach to handling missing data, assuming that data is missing at random (MAR). To examine whether this assumption may have been met, prior to running the primary analysis, we examined patterns of missing data via pattern-mixture models.66 The LMM included fixed effects of time, group, and a group x time interaction, with random intercepts of subject and site, as well as a random slope of time. Statistical significance was assessed at a 2-sided P-value of P < .05. We applied a Bonferroni adjustment to correct for multiple comparisons. Thus, the threshold for statistical significance was set at .05/2 = .025.

Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


Results

Participants and Baseline Characteristics

A CONSORT diagram of enrollment and allocation is shown in figure 1. Recruitment began in April 2015; the final participant completed post-training assessment in February 2018.

Demographic characteristics are presented in table 1 and medication regimens are shown in supplementary table 1. Groups were well-matched on demographic characteristics and estimates of premorbid intellectual abilities. Given the baseline differences between the groups on primary outcomes, we implemented propensity score weighting (supplementary material 2), which reduced the median standardized difference between groups from d = .11 at baseline to d = .03 after weighting.

Compliance and Feasibility of the SocialVille Training Program

Attrition Rate From the SocialVille Group. As can be seen in figure 1, 21 of the 76 participants (27.6%) randomized to the SocialVille group and 18/71 (23.6%) randomized to the control group dropped out from the study. There was no difference between study sites in terms of dropout rates (x2 = 4.67, P = .198). Those who did not provide subsequent data following the baseline assessment did not differ in terms of age, gender, education, race/ethnicity, or on the UPSA-2 (all P > .16). However, they performed moderately worse on the total SC composite (x2 = 5.02, P = .025, Cohen’s d approximation = .47). This suggests that our results reflect findings based on a slightly less severe population. There was minimal missing data among subjects that returned for follow-up assessments (10%), and sensitivity analyses via pattern-mixture models did not suggest that patterns of missing data influenced results among these subjects.

Most participants who dropped out from the study were unresponsive to calls made to them (n = 11) or just stopped training without providing a reason (n = 8). Additional reasons for dropping out were: difficulty completing training (n = 5), technological difficulties (eg, Wi-Fi problems; n = 3), medical complications unrelated to SZ diagnosis (n = 3), moving or change of job (n = 2), stopped attending the clinic (n = 2), did not train enough before completion of study (n = 2) or worsening of symptoms (n = 1). Finally, 3 participants were withdrawn by the investigators.

Compliance With Training. Participants were asked to complete 40 training sessions over 8-12 weeks. On average, all participants randomized to the SocialVille group completed 27.2 ± 13.3 daily sessions, and participants in the control group completed 25.5 ± 12.5 daily sessions (t = -0.94; P = .34).


Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


Table 1. Demographic and Clinical Characteristics for Socialville (SCV; n = 76) and Control (CTRL; n = 71) Groups

SCV Group (n = 76)

CTRL Group (n = 71)

t or X2

P Value

Age (y)

42.5 (13.9)

43.27 (11.5)

.05

.82

Male (n, %)

53 (69.7)

49 (69)

.01

.92

Education (y)

13.39 (2.3)

13.08 (1.64)

.12

.73

Age at diagnosis (y)

23.5 (9.5)

23.2 (9.2)

.05

.82

Race, n (%)

Caucasian

36 (47.4)

38 (53.5)

.56

.46

African-American

30 (39.5)

29 (40.9)

.03

.86

Other

10 (13.2)

4 (5.6)

2.41

.12

IQ Estimate (WTAR)

98 (11.46)

97.5 (11.46)

.10

.75

PANSS

PANSS Positive

15.14 (4.84)

14.82 (5.34)

.24

.62

PANSS Negative

16.96 (6.28)

16.23 (5.96)

.38

.54

PANSS General

30.05 (7.78)

30.23 (7.89)

.03

.86

PANSS Total

62.16 (15.02)

61.27 (15.55)

.218

.64

Baseline SC and functional capacity

Baseline SC composite

97.68 (14.98)

102.67 (14.66)

4.14

.042*

Baseline UPSA-2

34.18 (6.33)

36.45 (6.42)

4.89

.027*

Note: Means and SEMs are given. PANSS, Positive and Negative Syndrome Scale; SC, social cognition; UPSA-2, UCSD Performancebased Skills Assessment; WTAR, Wechsler Test of Adult Reading.

*Values are significant (P < .05).


Table 2. Intent-to-Treat Analysis Using Linear Mixed-Effects Models

SC Composite                                     UPSA-2

Variable

B (SE)

\z\

P

95% CI

B (SE)

\z\

P

95% CI

Time

1.78 (.48)

3.70

< .001

.84, 2.73

1.33 (0.49)

2.69

.007

0.36, 2.30

Group

-4.08 (3.98)

1.03

.305

-11.89, 3.72

-1.78 (1.44)

1.23

.217

-4.60, 1.05

Time x Group

2.81 (.49)

5.78

< .001

1.86, 3.76

-0.14 (0.56)

0.24

.808

-1.24, 0.97

Note: SC, social cognition; UPSA-2, UCSD Performance-based Skills Assessment.


Study completers completed 33.8 ± 8.1 and 29.6 ± 9.5 daily sessions of SocialVille and control training, respectively.

Primary Outcomes

Growth Models. Likelihood ratio tests and Bayesian Information Criterion indicated that linear trajectories of time fit the data best for both co-primary outcomes. In terms of unconditional growth models, there was a significant increase in total SC composite scores over time (b = 3.33, |z| = 6.10, P < .001). Effect size estimates (the estimated effect size for within-subject change over time was defined as where A is the rate of change and 6a is the standard deviation of the rate of change. The group x time effect size was defined as (&scv/$&-scv ) (Ac/6^-c ), where SCV refers to the experimental group and C refers to the control group) for within-subject change was .51 for the total composite score and .54 for UPSA-2 scores, which are in the “moderate” range.

Results of Intent-to-Treat Analyses. Findings from the ITT models are shown in table 2. There was a significant group x time interaction on the total SC composite score, such that participants in the experimental group exhibited greater change over time than control group, in the moderate-large range (estimated Cohen’s d = .65; figure 2, left). However, there was no group x time interaction on UPSA-2 scores (figure 2, right). There were no significant group x time x baseline performance interactions (all P > .08). Actual outcome values are provided in supplementary table 2.

We conducted additional analyses on the primary outcomes (supplementary material 2). We found that change on the UPSA was moderated by baseline UPSA-2 scores, and that change on the SC composite was correlated with change in SocialVille training ToM exercises. However, there was no association between compliance rates and changes on primary outcomes. In addition, a Principal Component Analysis (PCA) conducted on the primary SC outcome showed that the SC composite had 2 components and that the component that included all outcomes except for memory for faces (PROID, ER40, MSCEIT, and Empathic Accuracy) was the main contributor to the group x time interaction.

Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020



Secondary Outcomes

Secondary outcomes were examined using the same analytic strategy outlined above. Results were corrected for multiple comparisons and P-values were adjusted using the Benjamini-Hochberg False Discovery Rate (FDR). These results are summarized in table 3.

Significant group x time effects favoring the experimental group were found for the SFS - withdrawal subscale (b = 1.17, |z| = 6.68, P < .001), and for a facial affect recognition task (Morphed Faces Task; b = 0.02, |z| = 2.9, P = .004). However, the SFS interpersonal communication subscale showed an effect favoring the control group (b = -2.42, \z\ = 5.48, P < .001).

In addition, significant effects favoring the experimental group, but that did not survive the FDR correction, were found on a functional capacity outcome (VRFCAT; b = -57.53, |z| = 2.24, P = .025), the GFS - Social (b = 0.25, |z| = 2.62, P = .009), the QLS - motivation subscale (b = 0.19, |z| = 2.23, P = .026) and the TASIT - TO DO subscale (b = 0.15, |z| = 2.58, P = .01). Significant effects favoring the control group which did not survive FDR correction were found only for the SFS independent competence subscale (b = -1.17, |z| = 2.34, P = .019).

Discussion

We conducted an RCT to test the efficacy of an online SC training program in outpatients with schizophrenia. Our results show that, compared to an active control condition, SocialVille group participants showed greater improvement on independent behavioral composite measures of SC. The improvement on the functional capacity outcome (UPSA-2) was similar in both groups. However, only the experimental group showed significant improvements on the secondary outcomes of SFS - withdrawal and on a facial affect recognition measure. In addition, SocialVille group participants improved on a VR functional capacity measure (VRFCAT), in social functioning (GFS - social), on the SC measure of TASIT (TO DO subscale) and on a clinician-rated motivation subscale of the QLS. No change was seen on symptoms or self-report measures of motivation.

These results add to those of previous studies of SocialVille, which showed SC benefits following training in a small, uncontrolled study33,34 and in a controlled study in young healthy adults.36 In addition, applying SocialVille in a combination with “cold” cognitive training34,35 or with other types of therapy67 yielded improved SC function compared with control intervention. The current study extends these results and allows for evaluation of this remotely administered, targeted SC intervention.

To our knowledge, this is the only RCT to report the results of an online SC training compared to an active control. Most reported SC interventions in schizophrenia were applied in group settings in the clinic, managed by clinicians.68 Computerized interventions to date are mainly computer-assisted (eg,69-71) and were applied in the clinic, supervised by trained personnel as part of group sessions within a broader context (eg,30,69,72). Only one other study used a remote online SC intervention of emotion perception and ToM (eMotion training).73 However, the study included treatment-as-usual rather than an active control, which makes it difficult to account for placebo effects.74

The effects found here are comparable to those reported in a recent review of manualized, group-based SC interventions,75 reporting medium-to-large effect sizes for affect identification, mentalizing, and social perception. Our results are similar to those reported for the eMotion training,76 which showed improvements in emotion recognition and some aspects of ToM. Collectively, these

Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


Table 3. Analysis of Secondary Outcome Measures

Domain (Test)

Variable

b

\z\

P-Value

Social Functioning

&SFS - Engagement/Withdrawal

1.17

6.68

.000*

SFS - Interpersonal Communication

-2.42

5.48

.000*

SFS - Independence Competence

-1.17

2.34

.019

SFS - Prosocial

1.04

0.81

.418

SFS - Recreation

0.70

0.80

.424

SFS - Employment

0.73

0.71

.478

SFS - Independence Performance

1.00

1.92

.055

Morphed Faces Task

&Total Accuracy

0.02

2.90

.004*

Global Functioning

&GFS - Social

0.25

2.62

.009

GFS - Role

0.07

0.33

.741

Functional Capacity

&VRFCAT Total

-57.53

2.24

.025

Quality of Life

&QLS - Motivation

0.19

2.23

.026

QLS - Role

0.05

1.19

.234

QLS - Anhedonia

-0.13

1.10

.271

QLS - Purpose

0.11

0.64

.522

QLS - Interpersonal

0.09

0.34

.734

QLS - Curiosity

-0.08

0.32

.749

QLS - Social Interaction

-0.02

0.15

.881

QLS - Empathy

-0.01

0.11

.912

QLS - Commonplace

0.00

0.04

.968

TASIT

&TASIT - TOTDO

0.15

2.58

.010

TASIT - TOTFEEL

0.21

1.03

.303

TASIT - TOTTHINK

-0.20

0.89

.373

TASIT - TOTSAY

-0.03

0.13

.897

Attribution Bias (AIHQ)

AIHQ - Item

.25

.71

.477

AIHQ - HB

.27

.48

.634

AIHQ - AB

.06

.29

.773

ToM (Faux Pas)

Faux Pas Total

0.00

0.05

.960

Source Memory Test

Average Hit Rate

0.00

0.10

.897

Motivation

TEPS - Total

-0.70

0.70

.459

BIS/BAS - BAS Total

0.25

1.20

.250

BIS/BAS - BIS

-0.15

0.52

.603


Note: AIHQ, The Ambiguous Intentions Hostility Questionnaire; BIS/BAS, Behavioral Inhibition/Behavioral Activation Scale; SFS, Social Functioning Scale; GFS, Global Functioning Scale; VRFCAT, Virtual Reality Functional Capacity Assessment Tool; QLS, Quality of Life Scale; ToM, theory of mind; TASIT, The Awareness of Social Inference Test; TEPS, Temporal Experience of Pleasure Scale.

P-values represent the group x time interactions. Those marked with * are those P values < .05 with FDR. Those marked in bold are significant (P < .05) but did not survive FDR. Those favoring the SocialVille group are marked with a preceding &.


results show that an individualized online program can drive benefits in standardized SC outcomes.

Our results regarding the benefit of training on functional capacity were mixed. On the one hand, there were no group differences in improvements in the co-primary outcome UPSA-2. This result is in line with the null effects found for UPSA-2 in SC intervention studies (refs.22,77,78; see also refs.26,27). This could be due to the nature of the assessment itself, which may not be sensitive to detected changes in performance, or due to its tasks being outdated relative to the large technological advances characterizing the modern world. Additional concurrent treatments may be required in order to drive generalizable real-world functioning benefits.79

On the other hand, only the experimental group improved on a novel measure of functional capacity, VRFCAT, although this effect did not survive FDR correction. The VRFCAT itself has shown good validity and correlation with standard tests of functional capacity and


with occupational status.80-82 These results are in line with studies showing a strong link between SC and functional outcome in schizophrenia.14

Analysis of secondary outcomes reveals a more complex picture. Significant effects were found for facial affect perception (Morphed Faces task) and for everyday social functioning (SFS - withdrawal subscale), but not for a written ToM assessment (Faux Pas) or an attributional bias questionnaire (AIHQ), both of which were reported to have small-moderate effect sizes in previous reviews.75 In addition, some outcomes were statistically significant, but lost significance following FDR correction due to the large number of outcomes, and hence should be interpreted with caution. These include social awareness (the “to do” subscale of TASIT), the GFS-social, and the motivation subscale of the QLS. This suggests that targeting SC deficits may improve some form of motivation be-havior,60,83 and that this improvement may translate to improved functioning.35


Downloaded from https://academic.oup.com/schizophreniabulletin/article-abstract/doi/10.1093/schbul/sbaa085/5866409 by Serials Section, Dixson Library user on 03 July 2020


This is the largest controlled SC training trial testing a fully remote intervention to date and the results support the efficacy of SocialVille as a viable means to improve SC in patients with established schizophrenia. Specifically, it addresses some of the limitations of previous SC intervention studies, including small sample size, lack of appropriate controls, limited blinding of assessors, and limitations of outcome measures.27 Still, our study has several limitations that should be controlled for in future studies. These include the lack of follow-up, which precludes us from inferring about durability of the effects. Furthermore, participants received monetary incentives for participation, which could serve as external motivation, making it difficult to infer about the usefulness of the intervention in the real-world. In addition, training was performed using loaned laptop devices, which may not be available for all patients in real life. The relatively high attrition rate (±30%) is similar to that seen in other SC training studies (eg, refs.23,69,84). However, it calls for improved methodologies to keep participants engaged and potential modification of training requirements. In addition, the fact that those who dropped out had worse SC performance at baseline may indicate that these people may have found training more challenging or difficult to complete. Training deployed online or on mobile devices may help increase usability, as training can be performed also outside the home setting.85,86

Our study importantly shows that an individualized online intervention may be efficacious in schizophrenia and may serve as an adjunct to psychosocial and/or pharmacological treatments. Surprisingly, despite the increase in use of computers and mobile devices by large segments of the population, there have been only a few attempts to develop a fully computerized SC intervention. The current study may, therefore, serve as another step in the direction of integrating computerized interventions as part of the therapeutic regime of patients with chronic schizophrenia.

Supplementary Material

Supplementary material is available at Schizophrenia Bulletin online.

Funding

Research reported in this publication was supported by the National Institute of Mental Health (NIMH) Award R44MH091793 and by the National Center for Advancing Translational Sciences of the National Institutes of Health (Award Number UL1-TR002494). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Acknowledgments

We would like to thank Dr. Maurice Fried, Abby Rowlands, Lisa Howard, Jessica Dow, Connie Ludwig, Martin Hanson, Shasteana Rancher, Ariel Currie, Riley Capizzi, Gabrielle Steinhoff, Chetana Guthikonda, Emily Johnson, Sarah Pridgen, Abhishek Saxena and Briana Galindo for their help with data collection. Authors H.L., M.M.M., and S-J.K. are paid employees and A.R. serves as a consultant and holds equity in Posit Science, the company which holds rights for the computerized SocialVille training, which is described in the manuscript.

References

259


British Journal of Clinical Psychology (2010), 49, 259-274

© 2010 The British Psychological Society


www.bpsjournals.co.uk

The efficacy of SMS text messages to compensate for the effects of cognitive impairments in schizophrenia

G. H. M. Pijnenborg1,4,633, F. K. Withaar1,5, W. H. Brouwer2,3,4,

2                            5                    7

M. E. Timmerman , R. J. van den Bosch and J. J. Evans 1Department of Psychotic Disorders, GGZ Drenthe, Assen, The Netherlands 2Department of Psychology, University of Groningen, The Netherlands 3Department of Neurology, University Medical Center Groningen, University of Groningen, The Netherlands

4School for Behavioural and Cognitive Neurosciences (BCN), University Medical Center Groningen, The Netherlands

5Department of Psychiatry, University Medical Center Groningen, University of Groningen, The Netherlands

6Neuroimaging Center, University Medical Center Groningen, University of Groningen, The Netherlands

7Section of Psychological Medicine, University of Glasgow, Academic Centre, Gartnavel Royal Hospital, Glasgow, UK

Background and aims. Many people with schizophrenia have severe cognitive impairments that hamper their activities. The effect of pharmacological and behavioural interventions on cognitive functioning has been demonstrated, but even after successful intervention considerable impairments can remain. Therefore, we sought for alternative ways to help patients cope with the effects of their cognitive impairments. In the present study, we have evaluated the efficacy of short message service (SMS) text messages to compensate for the effects of cognitive impairments in schizophrenia in daily life.

Design. Awaiting list controlled trial was conducted: patients were quasi-randomly assigned to an A-B-A (baseline-intervention-follow-up) condition or an A-A-B-A condition that included an additional 7-week waiting list. The waiting list was included to control for the effect of time on relevant outcome.

Method. Sixty-two people with schizophrenia or related psychotic disorders were included in the study. All patients showed impaired goal-directed behaviour in daily lifesituations. Patients were prompted with SMS text messages to improve their everyday functioning. The primary outcome measure was the percentage of goals achieved.

260 G. H. M. Pijnenborg et al.

Results. The overall percentage of goals achieved increased with prompting, while performance dropped to baseline level after withdrawing the prompts. Keeping appointments with mental health workers and carrying out leisure activities increased with prompting, while medication adherence and attendance at training sessions remained unchanged. A majority of the patients enjoyed receivingthe SMS text messages. Discussion. Prompting can significantly improve achievement of a number of relevant goals. For other goals, combining prompting with interventions that enhance motivation seems indicated.

Cognitive impairment is very common in schizophrenia; performance on neuropsychological assessment is within the normal range in only 20-30% of patients (Harvey & Keefe, 1997; Holthausen et al., 2002). A number of factor analytic studies have revealed eight cognitive domains that are often impaired in schizophrenia: speed of processing, attention/vigilance, working memory, verbal learning, reasoning and problem solving, verbal comprehension, visual learning, and social cognition. In many patients with schizophrenia these cognitive impairments severely limit goal-directed behaviour and thereby activities and participation in daily life (Mueser, 2000; Pijnenborg, Withaar, Evans, van den Bosch, Timmerman, & Brouwer, 2009). Psychiatric symptoms of schizophrenia, and negative symptoms in particular, have also been associated with poorer goal-directed behaviour in schizophrenia (e.g. Frith, 1992).

The beneficial effects of antipsychotic agents on both cognitive impairments and negative symptoms are limited (Purdon, 2000). Therefore, cognitive rehabilitation (CR) techniques, defined as interventions that are intended to ‘help patients with cognitive problems and their families to cope with their impairments, to learn to live with them, to overcome and/or to reduce them’ (Wilson, 1997, p. 117), have been applied in schizophrenia. The term ‘cognitive rehabilitation’ encompasses interventions where the aim is to restore normal, or near normal cognitive functions (e.g. by drill and practice of cognitive skills) and techniques where the aim is to compensate for cognitive deficits either through training mental strategies or through provision of external aids or other forms of environmental support to decrease the burden of cognitive impairments. Two meta-analyses concluded that although CR in schizophrenia has a small positive effect on neuropsychological test performance and on psychiatric symptoms, improved task performance does not generalize spontaneously to relevant situations in daily life (see Krabbendam & Aleman, 2003; Twamley, Jeste, & Bellack, 2003). The most recent metaanalysis of CR in schizophrenia shows that combining CR with concurrent rehabilitation programmes (e.g. job coaching or social skill training) does lead to improved psychosocial functioning (McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007). Nevertheless, even after relatively successful CR severe limitations on the ability to participate in activities of daily life persist.

The vast majority of studies included in these meta-analyses use drill and practice and strategy training. Environmental adaptation has been much less frequently studied in the CR of schizophrenia. This is remarkable given the extensive use of environmental adaptation, most often in the form of external aids (assistive technology designed to support functioning in everyday life in people with cognitive impairment) in patients with acquired brain injury (e.g. Evans, Emslie, & Wilson, 1998; Wilson, Emslie, Quirk, & Evans, 2001). The lack of studies on the use of external memory aids in schizophrenia is a significant omission, given the resemblance of their cognitive impairments to those of patients with traumatic brain injury (TBI) and the effectiveness of such aids with the latter (Wilson et al., 2001).

SMS as cognitive aid in schizophrenia 261

Systematic reports on external memory aids or environmental adaptation in schizophrenia are very few in number. Three studies have examined whether prompting would lead to an increase in attending appointments in schizophrenia (see for a review, Reda & Makhoul, 2001). In two studies, patients received a reminder telephone call 24 h (Kluger & Karras, 1983) or 48 h (Burgoyne, Acosta, & Yamamoto, 1983) before clinic appointments. Attendance increased by phoning patients a day before appointments. Short written reminders a few days before the appointment day, also increased clinic attendance (Kluger & Karras, 1983; Swenson & Pekarik, 1988). In another study of environmental adaptations, Velligan (e.g. Velligan et al., 2006) used a combination of external aids, such as alarms, and labels to bypass executive problems. The intervention led to an improvement of everyday functioning and medication adherence (Velligan et al., 2008).

The results of these studies were promising. Therefore, we decided to build upon them and studied the efficacy of a cognitive prosthesis in the form of short message service (SMS) text messages. SMS text messages are short text messages (up to 160 characters), sent to mobile phones. The extensive use of mobile telephones in the general population makes the use of this kind of cognitive prosthesis non-stigmatizing and most schizophrenia patients are likely to be familiar with them. A pilot study (Pijnenborg, Evans, Withaar, van den Bosch, & Brouwer, 2007) showed promising results in a small case-series.

In the present study, the efficacy of SMS text messages in prompting behaviours was evaluated. In the same way that spectacles can be said to compensate for visual impairment but do not lead to enduring improvement in vision when they are removed, we predicted that SMS text messages would lead to an increase in the percentage of goals achieved in daily life, followed by a decrease in performance after their withdrawal. In addition to this primary hypothesis, we were interested in whether patients who benefit from the SMS text messages would also improve on indirect outcome measures of self-esteem, community functioning, and psychiatric symptoms. Moreover, we examined the predictive validity for success of a number of variables. Finally, we were interested in whether patients felt positive about the intervention and whether or not they would want to continue to use the SMS text message prompting afterwards.

Methods

Patients

Inclusion criteria for the study were broad: all patients that received care in the Department of Psychotic Disorders of GGZ Drenthe with observed limitations in goal-directed behaviour in daily life could participate in the study. Patients who are treated at the Department of Psychotic Disorders all have a diagnosis in the schizophrenia spectrum or an axis 2 diagnosis with psychotic symptoms according to the Diagnostic and Statistical Manual of Mental Disorders IV (DSM IV) criteria.

Sixty-two people (49 men, 13 women) participated in the study. They were diagnosed with schizophrenia (N = 53), schizoaffective disorder (N = 4), schizotypal personality disorder (N = 2), psychotic disorder NOS (N = 3) according to DSM IV criteria. Diagnoses were determined using chart information and confirmed by independent onsite psychiatrists involved in the treatment of these patients.

Patients mean age was 28.8 years (SD 8.8). A scale ranging from 1 = primary school to 7 = university (Verhage, 1983) was used to classify the level of education; the mean 262 G. H. M. Pijnenborg et al.

level of education was 4.5 (SD 0.9). The mean number of psychotic episodes was 1.6 (SD 0.97). Four patients did not use antipsychotic medication; two patients used a combination of conventional antipsychotic medication (penfluridol, chlorprothixene) and atypical antipsychotic medication (olanzapine, risperdone). The remaining patients used atypical antipsychotic medication (aripripazole, N = 2; clozapine, N = 16; olanzapine, N = 17; quetiapine, N = 3; and risperdone, N = 18). In addition, a number of patients used anti-depressive medication (N = 23), lithium (N = 3), and /or benzodiazepines (N = 24).

Eight patients were living independently, seven of them received out-patient care, and one participated in a rehabilitation programme. Six patients were living in sheltered housing. The remaining 48 patients were in-patients, 41 of them were following an in-patient rehabilitation programme.

Materials

SMS text messages

Thirty patients were provided with a Nokia 8310 or 8210 during the intervention, while 24 patients used their own mobile telephone. SMS text messages for each patient were entered and sent via a web application built for the purpose of the study.

Achievement of goals

Several mental health workers and family members that interacted frequently with the participants were instructed to observe patient’s behaviour and to score whether goals were achieved on score forms that were developed for each goal of each patient. Each goal was observed by one observant. Behaviour was scored binary: successful when the goal was achieved within a specified time frame (e.g. medication taken within 1 h of the planned time) or non-successful if the goal was achieved too late or was not achieved at all.

Tests and questionnaires

Cognitive functioning. To measure cognitive abilities we used: the short version of the Groninger Intelligence Test (Luteijn & van der Ploeg, 1983); a Verbal Memory Test (15 Words Test; Saan & Deelman, 1986); a test of behavioural memory (Rivermead Behavioural Memory Test; Wilson, Cockburn, & Baddeley, 1989); a Vigilance Test (Continuous Performance Test; Cornblatt & Keilp, 1994); a test of planning (Six Elements Test, SET; Burgess et al.,1996); a test of perceptual-motor speed and mental flexibility (Trail Making Test; Reitan, 1979); a Theory Of Mind Test (Faux Pas Test; Stone, Baron-Cohen, Calder, & Keane, 1998); a test of perception of emotional prosody (Prosody Test; Pijnenborg, Withaar, van den Bosch, & Brouwer, 2007); and a test of facial affect perception (Facial Expression of Emotion: Stimuli and Tests, FEEST; Young, Perrett, Calder, Sprengelmeyer, & Ekman, 2002).

Motivation. A shortened version of the Client Motivation for Therapy Scale (Deci & Ryan, 1985) was used to assess motivation. Nine items that measure motivation for a specific intervention on a five-points scale were selected from the original questionnaire.

SMS as cognitive aid in schizophrenia 263

Psychiatric symptoms. The positive and negative syndrome scale - interview (PANSS; Kay, Fishbein, & Opler, 1987) was used to measure psychopathology. Symptom clusters were based upon the model of Lindenmayer, Bernstein-Hyman, and Grochowski (1994), which encompasses a five-factor structure of psychiatric symptoms consisting of positive symptoms, negative symptoms, disorganization, depression, and excitement.

Social functioning. The Social Functioning Scale (Birchwood, Smith, Cochrane, Wetton, & Copestake, 1990) was used to assess social community functioning.

Self-esteem. The Rosenberg Self-Esteem Questionnaire (Rosenberg, 1965), a 10-item self-report questionnaire.

Procedure

Impaired performance on neuropsychological assessment does not have a one-to-one relationship with actual limitations in naturalistic settings. Although there is a moderate association between test performance and activities in daily life, schizophrenia patients with impaired performance on neuropsychological tests can still perform certain activities in daily life adequately and vice versa. Our intervention was aiming to improve functioning in daily life. As the intervention should be tailored to the level at which impairments become manifest, selection of patients for the study was based on observed limitations in daily life. Patients were referred to the study by nursing staff, psychologists, or psychiatrists when impaired goal-directed behaviour (e.g. frequent failures to attend appointments or poor medication adherence) was observed. All patients gave written informed consent. Subsequently, cognitive functioning, psychiatric symptoms, social community functioning, and self-esteem were assessed by independent raters. After assessment patients attended six weekly sessions of 1 h in groups of five to seven patients. The first session contained information about the project, the following three sessions included psycho-education on the effects of cognitive impairments in schizophrenia and ways to deal with them; in the final two sessions patients were trained in reading SMS text messages. As psycho-education was offered before the first measurement of goals, any effect of psycho-education on behaviour should have been already present during baseline and should therefore not affect the difference between baseline and intervention.

Thereafter, goals for the intervention were set. The intervention was tailored to individual needs by encouraging patients to choose their own goals. By doing so, we tried to exclude the possibility that patients were not motivated for the goals they did not achieve. Some patients had difficulties reporting on their daily behaviour or in setting realistic goals, so a nurse or family member who interacted with the participant on a regular basis was present to assist the participant whenever necessary. These persons made suggestions of goals whenever the patients were not able to identify goals themselves. For each patient, a schedule of SMS text messages was developed and entered into a website. For each goal, two prompts were sent. The first was sent an hour before the goal behaviour should take place, to enable patients to fit the target action into their current schedule. In a previous pilot study, patients sometimes arrived late at prompted sessions when a 10 min prompt was used because they were engaged in other activities and needed more time to end current activities and reach the correct location.

264 G. H. M. Pijnenborg et al.

A second prompt was provided 10 min before goal behaviour was due, so patients could then initiate relevant actions (e.g. walking to the consulting-room of their psychiatrist).

The most common goals were ‘taking medication’, ‘appointments with mental health workers’, and ‘attending the training programme’. Other goals were activities such as ‘grocery shopping’ or ‘attending a band rehearsal’. Two patients asked for prompts to inhibit behaviour rather than activations; their goals were ‘not eating more than one portion of dinner’ and ‘relaxing two hours during the afternoon’.

The first 33 patients that were referred to the study followed an A1-B-A2 design, with A1 being the baseline phase, B being the intervention phase, and A2 the follow-up phase. For the rest of the patients, an extra baseline condition was added to control for the effects of time passing; 29 patients were assigned to a waiting list control condition, resulting in an A1-A0-B-A2 design. The assignment of patients to conditions can be considered as quasi-random, because the assignment was based on the availability of places in the experimental treatment condition and not in any sense on patient characteristics. It was not possible for the assessors to be blind for conditions. Because psycho-education was offered in groups, patients commenced the trial in groups of five. It was of course, not possible for patients to be blind to condition.

A 2-week baseline (A1) for goal behaviours was set for all patients. During the baseline period the number of goals patients achieved in their daily life, without intervening, was measured. During this phase, all patients received care as usual. Subsequently, patients in the direct intervention group were prompted with SMS text messages during an intervention period of 7 weeks in addition to their usual care. In the last 3 weeks of the intervention period the number of goals they achieved was scored again (B). At the end of this period, assessment of social community functioning, psychiatric symptoms, and self-esteem was repeated. Three weeks after SMS text messages stopped, the number of goals achieved was scored again over a period of 2 weeks (A2). After completing the entire trial, patients filled out an evaluation form to measure subjective satisfaction with the SMS-prompts. Patients in the waiting list group were on a 7-week waiting list after the first baseline measurement (A1). The waiting list was added to the design to control for the effects of time passing; patients received usual care in this phase. During the last 2 weeks of the waiting list (A0) behaviour was scored while the rest of the trial was the same as in the A1-B-A2 condition.

Results

Figure 1 is a flowchart of the number of participants and drop-outs in each phase of the trial per condition. Reasons for drop-out are presented in Table 1.

Relevant demographic and clinical variables and number of goals at baseline for patients in each condition are summarized in Table 2. There were no significant differences between the direct intervention and the waiting list group on any of the relevant variables, except that patients in the waiting list condition performed poorer on the 15 Words Test and had more negative symptoms at a trend level (p = .051). These differences will be controlled for in further analyses.

Post hoc inspection of patients’ goals revealed that they could be divided into five categories: ‘medication adherence’, ‘appointments’, ‘activities’, ‘attending training sessions’, and ‘inhibition of undesired behaviour’. The number of goal categories within each patient varied from one to four. The number of times a specific goal behaviour

should occur varied considerably between goals and also between patients (see Table 2). Therefore, the percentage of goals achieved was used as the main outcome measure.

Between-group differences

First, we examined whether just putting people on a waiting list for 7 weeks had an effect on the overall percentage of goals achieved, and whether the patients in the two conditions differed at their first phase of the study. To this end, we compared the percentages between the first two phases of the study across the two conditions, i.e. A1-A0 of the waiting list group and A1-B of the direct intervention group. All patients in the waiting list condition who completed the relevant phases of each condition were included, regardless of whether they dropped out later on. Mean percentage success at A1 and A0 for the waiting list group were 55% (SD 27.4) and 52% (SD 29.5), respectively, and at A1 and B for the intervention group 47% (SD 28.2) and 67.1% (19.5), respectively.

To assess the significance of these differences we used logistic multi-level modelling (e.g. Snijders & Bosker, 2000) with the condition and phase as levels. The dependent

Table I. Reasons for drop-out

Baseline Loss of motivation to participate (N = 3)

Never showed up again after giving informed consent (N = 2)

Waiting list Continued treatment elsewhere (N = 1)

Did not want to participate anymore (N = 2)

Intervention Was not able to open SMS text messages due to disorganization (N = 1)

Sold the mobile telephone (N = 1)

Change of work situation in combination with exclusively work related goals (N = 1) Was annoyed by SMS text messages (N = 1)

Was unable the achieve goals due to back pain (N = 1)

Continued treatment elsewhere (N = 1)

Was not able to participate due to increased psychotic symptoms (N = 1)

Follow-up Continued treatment elsewhere (N = 3)

266 G. H. M. Pijnenborg et al.

Table 2. ANOVA for demographic characteristics, number of goals, and baseline performance of patients in the direct intervention condition and waiting list condition

Mean (SD)

Direct intervention (N = 30a) Waiting list (N = 21)

F

Age

28.3 (8.0)

25.7 (7.5)

1.3

Gender (percentage of men)

77

86

0.64b

Level of educationc

4.6 (0.8)

4.4 (1.0)

0.66

Number of episodes

1.7 (1.1)

1.6 (0.9)

0.16

Number of goals

20.9 (15.4)

21.1 (22.6)

0.01

Groninger Intelligence Test

83.3 (17.3)

N = 28

90.8 (18.6) N = 21

2.1

15 Words Test (number of words)

35.1 (10.4)

N = 28

41.2 (10.2) N = 21

4.2

Continuous Performance Test (d')

3.3 (0.85)

N = 27

3.7 (0.97) N = 20

2.9

FEEST (number of correct answers)

44.4 (6.3)

N = 27

47 (7.5) N = 21

1.7

Six Elements Test (total score)

5.2 (1.5)

N = 25

4.4 (1.8) N = 21

2.5

Rivermead Behavioral Memory Test

9.5 (3.1)

N = 27

8.1 (2.3) N = 21

3.0

(screening score)

Faux Pas Test (number of

4.3 (1.0)

N = 26

4.3 (1.1) N = 20

0.01

correctly identified Faux Pas)

Prosody (number of correct answers)

14.5 (3.7)

N = 25

15.0 (2.9) N = 20

0.27

Rosenberg self-esteem (total score)

29.1 (6.8)

N = 28

26.2 (5.4) N = 18

2.4

PANSS positive

14.1 (4.6)

N = 29

15.2 (6.3) N = 21

0.44

PANSS negative

14.1 (4.7)

N = 29

17.1 (5.1) N = 21

4.0

PANSS general

30.1 (7.0)

N = 29

31.8 (7.7) N = 21

0.42

Social Functioning Scale total

762 (53.2)

N = 29

754 (58.7) N = 21

0.20

Short Client Motivation for

19.5 (5.7)

N = 28

19.6 (4.6) N = 20

0.01

Therapy Scale

Note. Effects in italics p < .05.

aOne patient in the control condition never showed up for assessment, one was not testable. Furthermore, a number of patients only completed a number of tests because the entire battery was to demanding for them. Therefore, the number of patients of assessed patients is specified for each test. b A chi-square test was performed here.

c Ranging from 1 = primary school to 7 = university (Verhage, 1983).

variable was the overall proportion of goals achieved. In the multi-level model, the statistical significance of the regression effects was tested using the approximate t test. Dummy variables were used to indicate the phase (0 = phase 1,1 = phase 2) and the condition (0 = waiting list group; 1 = direct intervention). The dummy variables and their interaction were entered as fixed effects in the model. Furthermore, to account for possible effects of initial differences across groups, the negative symptoms of the PANSS and the performance on the 15 Words Tests were entered as fixed effects. As random effects, the between-individual and within-individual variances were estimated. All models were built using the program MlwiN.

The estimated coefficients and standard errors of the resulting logistic multi-level model are depicted in Table 3. The effects of phase and condition appeared to be nonsignificant, with relatively large standard errors. This implies that neither evidence for improvement during the waiting list period, nor for differences in initial performance between the two conditions are found. The interaction between phase and condition shows a significant improvement during intervention of the

SMS as cognitive aid in schizophrenia 267

Table 3. Logistic Multilevel Model of the effect of SMS text messages on overall percentage of goals achieved

Effect

Coefficient (standard error)

Fixed

Intercept

0.08 (.30)

Phase

- .12 (.33)

Condition

- .20 (.40)

Phase X condition

1.07 (.43)

Total 15 Words Test

.77 (.34)

PANSS negative

.02 (.01)

Random

Subject

.76 (.29)

Measurement

.60 (.21)

Note. Effects in italics p < .05. phase (0 = phase 1,1 = phase 2) and the condition (0 = waiting list group; 1 = direct intervention). The intercept refers to phase 1 of the waiting list group.

direct-intervention group. Thus, prompting with SMS text messages lead to an increase in success percentage, while performance remained relatively stable for patients in the waiting list did not change.

Main effects of SMS text messages

Results indicate that putting people on a waiting list had no clear effect on success percentage, as the performance remained stable over time without intervening. Therefore, we decided to combine both conditions for further analyses. This increases the study’s statistical power considerably, as most of the patients in the waiting list condition completed the intervention afterwards. Thus, the following are based on the entire group of 47 patients who completed the trial. The A-B-A design we use here is based on a previous study on the efficacy of a cognitive prosthesis in the CR of TBI (Wilson, Evans, Emslie, & Malinek, 1997).

Overall effect

The overall mean success percentage over all goal categories was 47% (across patients SD 27.9) during baseline, increased to 62% (SD 20.1) during the intervention, and dropped to 40% (SD 31.7) at follow-up (see Table 4).

Table 4. Mean success percentages (and standard deviation across patients) for medication adherence, attending training programme, individual appointments, activities, and inhibition

Total

Appointments

Medication

Training programme

Activities

Inhibition

Baseline

47 (27.9)

39% (32.1)

57% (28.8)

49% (39.6)

33% (29.5)

89% (15.7)

N = 38

N = 24

N = 14

N=6

N=2

Intervention

62% (20.1)

65% (26.2)

65% (25.3)

51% (35.1)

76% (23.8)

90% (14.1)

N = 38

N = 24

N = 14

N=6

N=2

Follow-up

40 (31.7)

56% (37.5)

48% (33.4)

37% (38.7)

25% (39.5)

67% (25.2)

N = 25

N = 19

N = 13

N=6

N=2

268 G. H. M. Pijnenborg et al.

To assess whether the differences between percentage success during baseline, intervention, and follow-up were statistically significant, logistic multi-level modelling was used. The dependent variable was overall proportion of goals achieved. An adequate representation of the variance structure of the repeated assessments was found using dummy variables for baseline and follow-up, taking the intervention as the reference category. The dummy variables were coded such that each parameter expresses the change between the intervention and the other measurement concerned. As random effects, the between-individual and within-individual variance were estimated.

Estimated coefficients and standard errors of the logistic estimated multi-level model for the overall successes are depicted in column 2 of Table 5 (model A). During the intervention, patients’ mean percentage success was significantly higher than during baseline (effect of MBaseline), and this increase was not maintained after withdrawing the SMS text messages (effect of MFollow-up).

Furthermore, we were interested in whether the effect of the intervention would be the same over goal categories. Mean percentage success for each of each phase of each of the categories was calculated (see Table 4).

Logistic multi-level models were build for the categories ‘appointments’, ‘medication’ and ‘training programme’ separately, using the same strategy as for the overall proportion success. The number of patients in the categories ‘activities’ and ‘inhibition’ were too small to perform further significance testing.

The estimated coefficients and standard errors of the resulting logistic estimated multi-level model are depicted in columns 4-6 of Table 5.

Appointments

As can be seen in column 4 of Table 5, during the intervention, patients’ mean percentage success was significantly higher than during baseline (effect of MBaseline), and was not maintained after withdrawing the SMS text messages (effect of MFollow-up).

Table 5. Logistic Multilevel Model of the effect of SMS text messages on overall percentage of goals achieved for separate categories

Effect

Dependent variable coefficient (Standard error)

Overall

Appointments

Medication

Training programme

Model A

Fixed

Intercept

.64 (.19)

.77 (.24)

.95 (.36)

- .23 (.41)

MBaseline

- .76 (.20)

- 1.04 (.26)

- .47 (.32)

- .51 (.33)

Mintervention

MFollow-up

- 1.20 (.21)

- .76 (.33)

- 1.11 (.35)

- .16 (.36)

Responder

Responder X Mintervention

Responder X MFollow-up Random Subject

.83 (.25)

.87 (.34)

1.79 (.67)

1.47 (.71)

Measurement

.56 (.14)

.30 (.23)

.69 (.25)

0 (0)

Note. Effects in italics: p < .05. Dummy codes for measurements: MBaseline (1 = baseline; 0 = otherwise); MIntervention (1 = intervention; 0 = otherwise); MFollow-up (1 = follow-up; 0 = otherwise); because of the dummy coding the intercept refers to the intervention phase for model A, and to the baseline phase for model B.

SMS as cognitive aid in schizophrenia 269

Medication

Although percentage success was 8% higher during the intervention than during baseline, this difference was not significant (effect MBaseline in column 5 of Table 5). After the intervention medication adherence dropped significantly (effect MFollow-up).

Training programme

The difference in the number of sessions of the programme training that were attended during each phase appeared not significant (effect MBaseline). The observed increase in goal-directed behaviour with prompting was only 2% (see column 6 of Table 5).

Activities and inhibition

Given the small number of patients in the categories activities and inhibition, we did not perform significance testing. For activities, the observed percentages success increased from 33 to 76% during the intervention and decreased to 25% afterwards. For inhibition, the success percentage was already high during baseline and did not increase further with prompting (both around 90%).

Predictors of success of prompting

To identify predictors of success of intervention, it is necessary to define success. Seventy-seven per cent of the patients performed better during the intervention than during baseline. However, as small changes may not be clinically significant, we decided to adopt a more stringent criterion of success. A 20% increase in overall percentage of goals achieved was used as the cut-off score to distinguish the groups, as we view a difference of at least 20% to indicate a clinically relevant difference. On the basis of this cut-off score, we divided the patients in two groups, responders and non-responders.

We investigated the difference in overall percentage of goals achieved between the two groups. Specifically, we investigated whether the responders and non-responders differed in their overall percentage of goals achieved at baseline. A logistic multi-level regression was used, with as predictors, the responder variable, the measurements, and their interaction. For clarity reasons, we refrain from presenting coefficients and standard errors but discuss the significance of the effects only. The non-responders (N = 25) showed a mean percentage success during baseline of 60% (SD 24.8) and 55% (SD 19.7) during the intervention. This means that the mean success percentage actually decreased somewhat with prompting in the non-responder group, but this decrease was not significant. In the responder group (N = 22), the mean percentage success increased significantly, from 34% (SD 24.6) at baseline and 71.5% (SD 17.3) during the intervention. The observed differences between responders and non-responders at baseline appeared to be significant: responders achieved significantly less goals during baseline than non-responders.

A set of variables that were expected to be associated with success of the intervention were selected: baseline assessment of cognitive functioning, psychiatric symptoms, and PANSS item G12 (insight), self-esteem, community functioning, and motivation.

Independent sample t tests were used to compare responders to non-responders. We found that non-responders performed significantly better than responders on the FEEST, 15 Words Test and had less positive symptoms (t = 2.5, p = .02; t = 2.11, p = .04; t = 2.11, p = .04; respectively). Furthermore, at a trend level non-responders also

270 G. H. M. Pijnenborg et al.

performed better on the SET (t = —2.11, p = .07). No significant difference on any of the other variables was found.

Effects on psychiatric symptoms, social community functioning, and self-esteem

To examine the effect of the intervention on secondary outcome, we performed paired sample t tests on the difference between psychiatric symptoms, social community functioning, and self-esteem. The t tests were performed in the responders and nonresponders separately, since we reasoned that possible indirect effects would be associated with an increased percentage success. Results are shown in Table 6.

Table 6. Effects of prompting on indirect outcome measures for responders and non-responders (baseline-follow-up)

Measure

N

respondersa

N nonresponders

Difference in mean (SD) responders

Difference in mean (SD) non-responders

t

responders

t nonresponders

Social Functioning

23

19

1.7 (35.7)

6.0 (43.9)

0.21

0.66

Scale (self)

Social Functioning

13

19

3.7 (41.9)

14.6 (58.8)

.32

1.7

Scale (other)

PANSS negative

21

22

2.2 (3.6)

1.0 (4.0)

2.7

1.1

PANSS excitement

21

22

0.57 (2.8)

0.68 (2.5)

0.95

1.2

PANSS disorganization

21

22

0.04 (3.2)

0.36 (3.3)

0.07

0.51

PANSS positive

21

22

0.23 (2.7)

0.32 (2.9)

0.40

0.52

PANSS depression

21

22

0.33 (3.3)

0.14 (3.6)

0.47

0.18

Rosenberg

20

23

0.60 (5.1)

1.4 (6.7)

0.52

1.0

Note. Effects in italics: p < .05.

aBecausea number of patients did not show up for (parts) of the follow-up assessment or were not able to fill out the questionnaires and a number of family members/nurses did not return the Social Functioning Scale the number of patients is specified for each measure.

Negative symptoms decreased in responders, whereas non-responders remained stable over time. Other secondary outcome measures remained unchanged.

Subjective evaluation

Forty-six patients filled out a brief evaluation form after the intervention. Thirty-two patients (70%) were positive, nine patients (20%) were neutral, while five patients (10%) were negative. Moreover, 19 patients (41%) thought the SMS text messages to be effective, 15 patients (33%) were neutral towards the efficacy of the SMS text messages, and 12 patients (26%) evaluated the SMS text messages as ineffective.

Finally, 22 patients (47%) were willing to continue the SMS text messages after the trial had stopped, 10 patients (22%) were not sure about continuation, and the remaining 14 patients (31%) did not want to continue. When data were split into responders and non-responders percentages did not differ across groups.

Discussion

To our knowledge this paper is the first to report on the efficacy of SMS text messages in the CR of schizophrenia. When prompted with SMS text messages, patients

SMS as cognitive aid in schizophrenia 271 achieved significantly more of their goals in daily life, although a considerable number of goals were still not achieved. The overall effect size of the prompting intervention was medium, and much higher than that of restorative and other compensatory approaches in the CR of schizophrenia (Krabbendam & Aleman, 2003; McGurk et al., 2007). It should be noted here that the effect sizes in these meta-analyses concerned improvement at a test level rather than in everyday functioning. When the effect of prompting on separate types of goals was examined, we found clear differences in the effectiveness of prompting. Results showed that prompting led to an increase in the percentage of appointments with mental health workers that patients attended, while there was also a favourable effect on leisure activities. However, prompting did not lead to a significant increase in medication adherence, attendance at the training programme, or the inhibition of undesired behaviour. It is worth noting that medication use does not reach sufficient mean frequency to be fully effective even during the intervention.

The fact that impaired goal-directed behaviour in schizophrenia is not entirely remediable by one simple intervention should not come as a surprise, since performance in schizophrenia is determined by many factors. This may explain why overall improvement in schizophrenia with prompting is more modest (about 10% less) than in patients with traumatic brain injury (Wilson et al., 2001). Besides cognitive impairments and negative symptoms, behaviour in schizophrenia is hampered by other symptoms of the disease, such as delusions and hallucinations. This is most often not the case in patients with traumatic brain injury. In addition, a number of goals in this study (taking antipsychotic medication or visiting training sessions together with other patients) may have been strongly associated with the stigma that accompanies schizophrenia or with undesirable side effects, which may have kept a number of patients from achieving them in spite of receiving a clear prompt. Patients may have been more motivated to achieve goals that were not associated with these costs and had clearer benefits (appointments with mental health workers). Observations of how patients chose their goals are consistent with this explanation. Post hoc inspection of our data showed that the majority of patients spontaneously asked for prompts for appointments, while activities were also spontaneously mentioned by a number of patients. Medication and training sessions were less often chosen by patients themselves and more often suggested by a nurse or family member. Given our results, it is to be expected that patients may have been compliant, but accepted goals for which they were not unambiguously motivated. The role of motivation and stigmatizing beliefs in the efficacy of prompting should be explored in future studies. Another reason for the fact that prompting did not lead to an increase in medication and training programme may be that groups for specific categories were relatively small and power was not large enough to detect change. Finally, the fact that the patients sometimes forgot to bring their phone or to load its battery will have inevitably limited the effect of the intervention.

The overall effect of prompting disappeared after the intervention. This suggests that efficacy of the SMS text messages is dependent on continuous use: as we expected there was no evidence of underlying cognitive functions being restored by the prompts. Alternatively, it may be that 7 weeks was too short to establish a routine and prompting over a longer period of time would lead to a lasting change in behaviour (though this might still be confined to the specific behaviour targeted). In the case of medication and training sessions, the success percentage dropped below baseline level during followup. It is not clear what accounts for this change, but may suggest that although

272 G. H. M. Pijnenborg et al.

performance did not improve with prompting, patients became familiar with the prompts and learned to rely upon them to some extent.

We also sought to determine predictors of success and distinguished patients that improved more than 20% (responders) from patients who did not. At baseline, responders achieved significantly less goals than non-responders. Moreover, nonresponders outperformed responders on test of memory, facial affect recognition and planning (at a trend level) and had more positive symptoms. Results suggest that the most severely impaired patients will benefit the most from prompting. Apparently, clear prompts compensate for poorer memory and planning abilities in responders and helps them to ignore distracting positive symptoms, such as hallucinations. The explanation for poorer affect recognition in responders is less straightforward. It may be speculated here that non-responders are more sensitive to non-verbal feedback by others and therefore are more motivated to be compliant with treatment, while for responders the prompts compensate for the lack of reinforcement in interpersonal interactions. Given the relatively small sample size these results should be considered with caution and their interpretation remains speculative.

The effect of the prompts on secondary outcome measures (self-esteem, psychiatric symptoms, and social community functioning) was also examined. Patients who profited from the intervention showed less negative symptoms after the intervention than during baseline. Probably that fact that these patients were activated by the prompts was also noticed during the PANSS interview. Alternatively, patients who had a reduction in negative symptoms for other reasons were better able to respond to the prompts. Contrary to our expectations, other indirect outcome measures were not changed by the intervention. It may be that the intervention was too short in time to bring about real changes in these outcome measures.

A subjective evaluation of the interventions by the patients showed that the majority of the patients was positive about the prompts and almost half of them wanted to continue after the trial had stopped. About half of the patients felt that prompting was effective to help them overcome their limitations.

There are a number of limitations to this study. First, patients in some categories were too few to do significance testing and as a result replication in a larger sample is warranted. Second, assessors were not blind to the research hypothesis, though as objective and unambiguous outcome measures were used, we do not feel that this has affected the study’s results. Third, patients were not randomized over conditions but referred to a given condition based on the timing of their inclusion. Finally, our sample is relatively homogeneous, containing both in-patients and out-patients. It should be noted here we used a quite liberal definition of in-patients by describing as ‘in-patients’ those patients that lived in houses provided by the institution in order to enable their participation in a rehabilitation programme. Most of these patients were able to retain their usual level of activities and roles in daily life and lived a life that was in many was comparable to that of the out-patients.

The results of our study have a number of implications for clinical practice. First, the decrease in performance after the intervention suggests that prompting over a longer period of time appears is necessary, at least for some behaviours. Second, if SMS text messages are to be used to enhance treatment adherence, they may need to be combined with interventions that enhance motivation for treatment, for example motivational interviewing (Kemp, Hayward, Applewhaite, Everitt, & David, 1996) or with interventions aimed to reduce stigmatizing beliefs. Many people with schizophrenia find it hard to associate long-term goals with behaviour in the

SMS as cognitive aid in schizophrenia 273 short term. For example, taking medication today may not be associated with a decrease of symptoms in a couple of weeks. Therefore, the effectiveness of prompting may be larger when short-term goals are explicitly associated with relevant outcome in daily life.

References

Birchwood, M., Smith, J., Cochrane, R., Wetton, S., & Copestake, S. (1990). The social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. British Journal of Psychiatry, 157, 853-859.

Burgess, P W, Alderman, N., Emslie, H., Evans, J. J., Wilson, B. A., & Shallice, T (1996). The simplified six element test. In B. A. Wilson, N. Alderman, P. W Burgess, H. Emslie, & J. J. Evans (Eds.), Behavioural assessment of the dysexecutive syndrome. Bury St Edmunds: Thames Valley Test.

Burgoyne, R. W, Acosta, F. X., & Yamamoto, J. (1983). Telephone prompting to increase attendance out a psychiatric outpatients clinic. American Journal of Psychiatry, 140(3), 345-347.

Cornblatt, B. A., & Keilp, J. G. (1994). Impaired attention, genetics, and the pathophysiology of schizophrenia. Schizophrenia Bulletin, 20(1), 31-46.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behaviour. New York: Plenum.

Evans, J. J., Emslie, H., & Wilson, B. A. (1998). External cueing systems in the rehabilitation of executive impairments of action. Journal of the International Neuropsychological Society4(4), 399-408.

Frith, C. D. (1992). The cognitive neuropsychology of schizophrenia. Hove: Erlbaum.

Harvey, P D., & Keefe, R. S. (1997). Cognitive impairment in schizophrenia and implicatons of atypical neuroleptic treatment. CNS Spectrums, 2, 1-11.

Holthausen, E. A. E., Wiersma, D., Sitskoorn, M. M., Hijman, R., Dingemans, P. M., Schene, A. H., et al. (2002). Schizophrenic patients without neuropsychological deficit: Subgroup, disease severity or cognitive compensation? Psychiatry Research, 112(1), 1-11.

Kay, S. R., Fishbein, A., & Opler, L. A. (1987). The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261-276.

Kemp, R., Hayward, P, Applewhaite, G., Everitt, B., & David, A. (1996). Compliance therapy in schizophrenia: RCT. British Medical Journal, 312, 345-349.

Kluger, M. P, & Karras, A. (1983). Strategies for reducing missed initial appointments in a community mental health centre. Community Mental Health Journal, 19(2), 137-143.

Krabbendam, L., & Aleman, A. (2003). Cognitive rehabilitation in schizophrenia: A quantitative analyses of controlled studies. Psychopharmacology, 169(3-4), 376-382.

Lindenmayer, J. P, Bernstein-Hyman, R., & Grochowski, S. (1994). Five factor model of schizophrenia initial validation. Journal of Nervous and Mental Diseases, 182(11), 631-638.

Luteijn, F., & van der Ploeg, F. A. E. (1983). Groninger intelligentie test (Manual). Lisse: Swets and Zeitlinger.

McGurk, S. R., Twamley, E. W, Sitzer, D. I., McHugo, G. J., & Mueser, K. T (2007). A meta-analysis of cognitive remediation in schizophrenia. American Journal of Psychiatry, 164(12), 1791-1802.

Mueser, K. T. (2000). Cognitive functioning, social adjustment and long-term outcome in schizophrenia. In T. Sharma & P. Harvey (Eds.), Cognition in schizophrenia: Impairments, importance and treatment strategie (pp. 157-177). New York: Oxford University Press.

Pijnenborg, G. H. M., Evans, J. J., Withaar, F. K., van den Bosch, R. J., & Brouwer, W H. (2007). SMS text messages as a prosthetic aid in the cognitive rehabilitation of schizophrenia. Rehabilitation Psychology, 52(2), 236-240.

274 G. H. M. Pijnenborg et al.

Pijnenborg, G. H. M., Withaar, F. K., Evans, J. J., van den Bosch, R. J., Timmerman, M. E., & Brouwer, W H. (2009). The predictive value of measures of social cognition for community functioning in Schizophrenia: Implications for neuropsychological assessment. Journal of the International Neuropsychological Society, 15(2), 239-247. doi:10.1017/S1355617709090341

Pijnenborg, G. H. M., Withaar, F. K., van den Bosch, R. J., & Brouwer, W. H. (2007). Impaired perception of negative emotional prosody. Clinical Neuropsychologist, 21(5), 762-775.

Purdon, S. E. (2000). Measuring neuropsychological change in schizophrenia with novel antipsychotic medications. Journal of Psychiatry and Neuroscience, 25(2), 108-116.

Reda, S., & Makhoul, S. (2001). Prompts to encourage appointment attendance for people with serious mental illness. Cochrane Database of Systematic Reviews. Issue 2 Art No: CD002085 doi:101002/14651858CD002085

Reitan, R. M. (1979). Manual of administration of neuropsychological test batteries for adults and children. Tucson, AZ: Neuropsychology laboratory.

Rosenberg, M. (1965). Society and adolescent self image. Princeton, NJ: Princeton University Press.

Saan, R. J., & Deelman, B. G. (1986). De 15-woordentests a (Manual). Groningen: University of Groningen, Department of Neuropsychology (intern publication).

Snijders, T. A. B., & Bosker, R. J (2000). Multilevel analysis. London: Sage.

Stone, V e., Baron-Cohen, S., Calder, A. W., & Keane, J. (1998). Impairments in social cognition following orbitofrontal or amygdala damage. Society for neuroscience abstracts, 1176.

Swenson, T., & Pekarik, G. (1988). Interventions for reducing missed initial appointments at a community mental health centre. Community Mental Health Journal, 24(3), 205-218.

Twamley, E. W., Jeste, D. V, & Bellack, A. S. (2003). A review of cognitive training in schizophrenia. Schizophrenia Bulletin, 29(2), 359-382.

Velligan, D. I., Diamond, P. M., Mintz, J., Maples, N., Li, X., Zeber, J., et al. (2008). The use of individually tailored environmental supports to improve medication adherence and outcome in schizophrenia. Schizophrenia Bulletin, 34(3), 483-493.

Velligan, D. I., Mueller, J. A., Wang, M., Dicocco, M. S., Diamond, P. M., Maples, N. J., et al. (2006). Use of environmental supports among patients with schizophrenia. Psychiatric Services, 57(2), 219-224.

Verhage, F. (1983). Het coderen van het opleidingsniveau voor researchdoeleinden (English translation: Educational classification system for research purposes: Revised version). Groningen: University Medical Center, State University Groningen, (Internal Publication).

Wilson, B. A. (1997). Cognitive rehabilitation: How it is and how it might be. Journal of the International Neuropsychological Society, 3(5), 487-496.

Wilson, B. A., Cockburn, J., & Baddeley, A. D. (1989). The rivermead behavioral memory test. Thurston: Thames Valley Test company.

Wilson, B. A., Emslie, H. C., Quirk, K., & Evans, J. J. (2001). Reducing everyday planning and memory problems by means of a paging system: A randomised control crossover study. Journal of Neurology, Neurosurgery and Psychiatry, 70(4), 477-482.

Wilson, B. A., Evans, J. J., Emslie, H., &Malinek, V (1997). Evaluation of neuropage: A new memory aid. Journal of Neurology, Neurosurgery and Psychiatry, 63(1), 113-115.

Young, A. W., Perrett, D. T., Calder, A. J., Sprengelmeyer, R., & Ekman, P. (2002). Facial expressions of emotion: Stimuli and test. Manual. Bury St Edmunds: Thames Valley Test Company.

Received 18 July 2008; revised version received 20 April 2009


American Journal of Psychiatric Rehabilitation


ISSN: 1548-7768 (Print) 1548-7776 (Online) Journal homepage: http://www.tandfonline.com/loi/uapr20


Routledge

Taylor &. Francis Group


A tablet-based intervention to manipulate social cognitive bias in schizophrenia

David L. Roberts, Philip Yen-Tsun Liu, Heather Busanet, Natalie Maples & Dawn Velligan

To cite this article: David L. Roberts, Philip Yen-Tsun Liu, Heather Busanet, Natalie

Maples & Dawn Velligan (2017) A tablet-based intervention to manipulate social cognitive bias in schizophrenia, American Journal of Psychiatric Rehabilitation, 20:2, 143-155, DOI: 10.1080/15487768.2017.1302897

To link to this article: http://dx.doi.org/10.1080/15487768.2017.1302897

Published online: 24 Apr 2017.

Submit your article to this journal S’



Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=uapr20

AMERICAN JOURNAL OF PSYCHIATRIC REHABILITATION

2017, VOL. 20, NO. 2, 143-155

http://dx.doi.org/10.1080/15487768.2017.1302897

A tablet-based intervention to manipulate social cognitive bias in schizophrenia

David L. Roberts, Philip Yen-Tsun Liu, Heather Busanet, Natalie Maples, and Dawn Velligan

Department of Psychiatry, UT Health Science Center at San Antonio, San Antonio, Texas, USA

ABSTRACT

KEYWORDS

cognitive bias; computer training; metacognition; schizophrenia; social cognition


Interventions to decrease cognitive bias in schizophrenia have yielded limited benefit. One reason may be that people receive insufficient rehearsal applying debiasing skills while biases are actively affecting their thinking. The authors designed Mary/ Eddie/Bill-internet (MEBi) to (1) teach debiasing skills to people with schizophrenia, (2) activate biases during training sessions, and (3) provide daily in-home rehearsal of debiasing skills using tablet computer interface. In this proof-of-concept trial, 28 adults with schizophrenia used the MEBi tablet "app" for one month. Fourteen completed a version of MEBi including only the debiasing skills, and 10 completed a version including a bias activation component. Participants completed pretest and posttest measures of social cognition and social functioning. Results showed that participants in both groups adhered to the intervention and learned the debiasing skills. Participants who were only taught the debiasing skills showed significant improvements in social cognitive bias, accuracy, and selfreported social functioning relative to participants who also received the bias-activation manipulation—who showed worsening social cognitive bias. Results suggest that it is feasible to affect social cognition in schizophrenia through in-home tablet-based training. However, more metacognitive training is needed to help people apply debiasing techniques when bias is activated.

Introduction

Social cognition is a promising treatment target in schizophrenia because it is a strong predictor of functional outcome (Couture, Penn, & Roberts, 2006; Fett et al., 2011). Most interventions have conceptualized social cognitive dysfunction as resulting from information-processing deficits. However, there is also evidence that social cognitive dysfunction in schizophrenia is partially attributable to bias processes (Green et al., 2008; Peer, Rothmann, Penrod, Penn, & Spaulding, 2004). Whereas information-processing deficit refers to the inability to perform an information-processing function, bias refers to the distorted or disregulated use of an intact information-processing ability.

CONTACT David L. Roberts @ robertsd5@uthscsa.edu Q Department of Psychiatry, UT Health San Antonio, 7703 Floyd Curl Dr., MC 7797, San Antonio, TX 78229, USA.

Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/uapr.

© 2017 Taylor & Francis

Two broad categories of bias have been identified in schizophrenia: a tendency toward overconfidence in one’s judgments (Evans, Averbeck, & Furi, 2015; Garety & Freeman, 1999; McKay, Langdon, & Coltheart, 2007; Woodward, Moritz, Cuttler, & Whitman, 2006) and a self-referential bias (Hooker & Park, 2005) that often includes inference of hostile thought or intention directed by others toward the subject (Combs, Penn, Micher, & Waldheter, 2007; Green & Phillips, 2004). Biological models of psychosis suggest that biases may result from dysregulation of amygdalar dopaminergic processing, leading to aberrant salience signaling to the frontal cortex (Kapur, Mizrahi, & Li, 2005; Winton-Brown, Fusar-Poli, Ungless, & Howes, 2014). Functionally, this may lead to (1) biasing of attention and attribution of self-relevant meaning to nonsignificant social cues and (2) overconfidence in one’s judgments about these cues (Rosenfeld, Lieberman, & Jarskog, 2011). Social psychological models posit that biases in schizophrenia result from self-serving motivated reasoning in which people maintain positive self-regard by attributing negative aspects of their life to the actions of nefarious external agents and minimize self-doubt by cultivating selfconfidence and avoiding disconfirmatory evidence (Bentall, Corcoran, Howard, Blackwood, & Kinderman, 2001; Garety & Freeman, 1999).

The majority of techniques for improving social cognition in schizophrenia focus on remediating deficits and have been adapted largely from neurocog-nitive remediation principles. These principles include strengthening implicit memory and attention skills through drill-and-repeat practice (Kurtz, Seltzer, Shagan, Thime, & Wexler, 2007), and also learning explicit strategies for using information (e.g., Wykes et al., 2007).

Techniques for decreasing bias are qualitatively different from these remediation techniques because they hinge on developing insight into one’s habits of distorted thinking and the causes of these habits. Debiasing techniques have been developed largely in psychotherapy research. In cognitive therapy, people learn to notice their own biased thinking by identifying and tracking their automatic thoughts, and then learn to modify thoughts that are dysfunctional (Beck, 2005). A difficulty with applying this cognitive approach is that social cognitive biases appear to be state-like rather than trait-like and may only emerge in the context of certain environmental, physiological, and emotional states. Thus, it is of limited value to practice debiasing techniques during treatment sessions if people are not currently experiencing bias exacerbations. However, it is also difficulty to practice debiasing during real-life situations between treatment sessions because people with schizophrenia often struggle to recall and skillfully deploy debiasing skills without the supportive prompts provided in the treatment environment. This problem is illustrated by the low adherence to homework assignments in Cognitive Behavior Therapy (CBT) for psychosis (Hogg & Hall, 1992).

One technique that may help people with schizophrenia to recall and use debiasing skills outside of session is to teach the skills in a simple, memorable format. For example, the consolidation of cognitive thought restructuring technique into the mantra “Catch it, check it, change it,” known as “the three C’s,” is widely used in CBT for psychosis because people find it easy to recall and use outside of session (Granholm, McQuaid, Auslander, & McClure, 2004). Alternatively, debiasing training can be enhanced by activating biases during session and then providing the person with support and rehearsal in using debiasing techniques. For example, in Metacognitive Training (MCT; Moritz, Woodward, Stevens, & Hauschildt, 2013), stimuli are used that lure people into making false judgments, and this is used to illustrate the importance of being able to flexibly change one’s judgments in social situations.

In the current study, we attempted to integrate both of these approaches— teaching an easy-to-remember heuristic, and activating biases during treatment sessions—into a tablet-based intervention for schizophrenia. Mary, Eddie, Bill (MEB) is a group intervention derived from a component of the Social Cognition and Interaction Training (SCIT) program (Roberts, Penn, & Combs, 2015). MEB teaches participants an easy-to-remember heuristic to decrease overconfidence in social judgments and improve theory of mind (ToM) (Roberts, Kleinlein, & Stevens, 2012). MEB-iPad (MEBi) teaches the same lessons using a tablet computer interface that increases opportunities for rehearsal. Additionally, we developed a version of the intervention called MEBi-Self that uses stimuli that are designed to activate self-referential bias. The current study is a pilot trial of the feasibility and potential efficacy of MEBi-Standard and MEBi-Self. We hypothesized that both interventions would be feasible and that MEBi-Self would have greater positive effects on social cognitive bias and accuracy than MEBi-Standard.

Method

Participants

Twenty-eight stable outpatient adults with schizophrenia spectrum disorders were recruited from community mental health centers in San Antonio, Texas. Diagnosis was confirmed with the SCID (First, Spitzer, Gibbon, & Williams, 1996). Participant demographics are summarized in Table 1.

Intervention conditions

Mary/Eddie/Bill-iPad (MEBi)

The MEBi app teaches participants that in any situation people’s thoughts and feelings usually can be categorized into one of three stereotypic character styles, each of which is associated with characteristic thoughts, feelings and behaviors: My-fault Mary (who is sad and self-blaming), Easy Eddie (who is happy and never blames others), and Blaming Bill (who is angry and always

Table 1. Baseline participant characteristics.

MEBi Standard (n =

10)

MEBi Self (n = 14)

M (SD)

M (SD)

Age

49.0 (9.6)

41.4 (10.6)

Education

12.6 (1.3)

13.3 (1.4)

#

#

Female

5

5

Male

6

8

Race/ethnicity

Black

2

3

Hispanic

5

5

White

3

6

Diagnosis

Schizophrenia

6

6

Schizoaffective

3

8

MDD w/psychosis

1

0

Symptoms

BPRS psychotic symptoms

10.5 (4.4)

10.3 (4.2)

BPRS negative symptoms

7.3 (2.1)

7.6 (2.4)

BPRS total

46.1 (12.7)

45.3 (10.9)

Cognition

HVLT Immediate Recall

20.5 (4.5)

23.6 (5.1)

Verbal fluency

31.9 (12.4)

34.4 (10.6)

Trails A

38.9 (13.9)

32.8 (10.0)

Trails B

83.1 (28.8)

95.4 (43.8)

MEBi = Mary/Eddie/Bill-iPad; MDD = Major

Depressive Disorder;

BPRS = Brief

Psychiatric Rating Scale;

HVLT = Hopkins Verbal Learning Test.

blames others). These characters are illustrated in tablet interface with colorful character descriptions and associated imagery and videos to make these prototypes easy to remember. In early sessions, participants use finger-stroke responding to practice categorizing people from photo, audio, and video stimuli as resembling Mary, Eddie, or Bill. Participants are then taught that it is not possible to know with certainty how others are thinking or feeling, and that it is socially risky to jump to the conclusion that a person is feeling like any of the three characters. To avoid overconfidence, participants practice flexibly applying all three prototypes to any person and any situation, and also to rate their confidence in their categorization responses (i.e., “How sure are you that the woman in this picture is feeling like Easy Eddie?” from 0% sure to 100% sure). Participants receive corrective feedback for rigid responding and for exaggerated confidence ratings. Thus, overall MEBi provides an easy technique for making quick guesses about others thoughts and feelings, along with a technique for flexibly considering alternate guesses.

Participants were asked to complete 24 15-minute MEBi sessions over one month (six sessions per week) in their home. One half (n = 14) completed the standard version of MEBi (MEBi-Standard) and one half (n = 14) completed a version designed to activate self-referential bias (MEBi-Self). MEBi-Standard, described above, comprised 12 sessions that focus on basic skills of flexibly categorizing photos and judging confidence in these categorizations. Photo, audio, and video stimuli all required third-person judgments—that is, judgments of strangers with no connection to the participant. Participants completed the 12 sessions twice, for a total of 24 sessions, to reinforce learning of basic concepts. MEBi-Self comprised 24 sessions in which the first 12 sessions were identical to MEBi-Standard and the latter 12 sessions incorporated increasingly self-referential stimuli and required introspection about reactions to social stimuli. Stimuli included actors who were looking out of the tablet screen toward the participant and speaking to the participant, followed by the participant being asked, “Which of the three characters (Mary/Eddie/Bill) did this person make you feel like?” and “How strongly did you feel that way?” MEBi-Self was designed to activate self-referential processing, leading to activation of motivated reasoning and social cognitive bias processes, to which the participant would, in turn, apply debiasing techniques.

Procedures

Fifty-two individuals were identified who met study criteria. Of these, 32 expressed interest, and 28 consented to participate after reviewing the consent agreement. After consent, participants completed baseline assessments with research staff, and then received 15 to 30 minutes of one-on-one training on the use of the MEBi tablet intervention. Participants were assigned to intervention condition chronologically, with the first one half receiving MEBi-Standard and the second one half receiving MEBi-Self. During the intervention period, participants’ progress on the MEBi app was monitored through remote connectivity to the tablet computers. Participants received regular phone check-ins from research staff to provide as-needed MEBi technical support and to provide positive reinforcement for adherence to the intervention.

Measures

Feasibility of the MEBi intervention was assessed in several ways. We tracked adherence to the in-home treatment through iPad finger-stroke activity and through participant feedback on posttreatment questionnaires. We measured participants’ memory at posttreatment for the MEB heuristic strategy to assess whether learning took place through the unsupervised in-home sessions. We also tracked whether tablet computers were lost or damaged.

At baseline, we measured participant demographics, we measured psychotic symptoms with the Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962), and we measured cognition with the Hopkins Verbal Learning Test—Immediate Recall, Verbal Fluency (letters), and the Trailmaking Making Test, parts A and B (Reitan & Wolfson, 1985).

Social cognition was assessed with four instruments to include self-relevant and non-self-relevant measures, and to assess accuracy and bias. Self-relevant measures are those in which the participant makes judgments about scenarios in which s/he is an actor. The Waiting Room Task (WRT) (Roberts & Hoffman, 2011) is a self-relevant measure that assesses social cognitive accuracy and bias domains. Participants view 26 brief videos simulating the experience of facing an unknown person in a waiting room. Across videos, the target person varies direction of gaze (at or away from the camera), duration of gaze (0.5 seconds vs. 2 seconds), and facial expression (angry, happy, afraid, neutral). For each video, the participant makes dichotomous judgments of gaze direction (whether the target person in the video looks directly at, or away from, the participant [i.e., camera]) and self-referential ToM (“Did it seem that s/he had a thought about you?”). Accuracy of gaze and ToM responses are summed with higher scores indicating greater accuracy (range = 0-52). Self-referential bias is measured as false alarm rate in endorsing self-directed gaze and self-directed thought (range = 0-1). The Social Cognition Screening Questionnaire (SCSQ) also is a self-relevant measure of accuracy and bias. It comprises 10 verbally administered vignettes that describe imaginary social interactions between the respondent and other people. All vignettes describe negative outcomes of ambiguous causality. After each vignette, respondents are asked whether a vignette character had negative thoughts or feelings toward the respondent. Responses on this self-referential ToM item are scored as correct or incorrect to yield an accuracy scale (range = 0-10, with higher scores indicating better accuracy). A hostile attributional bias scale is computed as the sum of ToM items on which the participant endorsed self-focused hostility when there was none (range = 0-5, with higher scores indicating more bias). An overconfidence bias score is computed from participants’ ratings of certainty in the accuracy of their judgments on the ToM items (“How sure are you that your answer is correct?”). The scale ranges from 0 to 10 with higher scores indicating greater overconfidence bias. The Hinting Task (Corcoran, Mercer, & Frith, 1995) is a non-self-relevant measure of social cognitive accuracy. It comprises 10 brief vignettes that describe imaginary third-person interactions in which one character utters an indirect verbal request of another character (a hint). Participants much infer the character’s intended meaning. The scale ranges from 0 to 20 with higher scores indicating better performance. Finally, As an additional measure of overconfidence bias, participants’ in-session ratings of confidence in their incorrect social judgments were recorded and averaged at mid treatment (Sessions 11 and 12) and at the end of treatment (Sessions 23 and 24) (range = 0-100, with higher scores indicating greater confidence in incorrect judgments).

Self-reported social engagement was assessed using the Relational Interactivity Measure (RIM; Ralph Hoffman, unpublished). a 5-item selfreport measure of the degree of social contact and connectivity experienced in the past week. Participants to estimate the number of rewarding and meaningful social interactions, total hours per day spent in the presence of others, and number of hours per day in which they actively chose versus avoided social contact. The scale ranges from 3 to 25, with higher scores indicating greater social engagement.

Data analysis plan

Feasibility data were analyzed using descriptive statistics. To compare the effects of MEBi-Standard to MEBi-Self, we conducted mixed ANOVAs with time (pre- vs. posttreatment) as a within-subjects factor and group (MEBi-Standard vs. MEBi-Self) as a between-subjects factor, using social cognition and social functioning as dependent variables.

Results

Participant characteristics are displayed in Table 1. The groups did not differ in demographics, symptoms, or cognitive domains.

Feasibility of the in-home tablet intervention

Of the 28 individuals who consented, 24 completed the in-home MEBi training regimen. Two participants withdrew, one citing difficulty with the tablet interface and the other citing lack of engagement in the intervention. (Two other participants were withdrawn by the principal investigator because symptom severity prevented ongoing participation.) Of the 24 completers, iPad finger-stroke data indicated that 19 (79%) had only minor deviations to adherence—such as one or two instances of failing to complete a training session on the scheduled day, and having to make it up the following day. Five participants (19%) deviated more (e.g., missing two or three consecutive days) but completed the full intervention within the one-month treatment period.

Participant feedback is displayed in Table 2. Responses were favorable on all measured domains. Participants reported that the app was easy to use and understand, was enjoyable, and helped them in thinking about and

Table 2. Participant feedback regarding the MEBi intervention (N = 24).

Question

Anchors 1 2 3 4 5 6 7

M

1) Was the app easy to use?

easy/hard

1.8

2) Were the three characters (Mary, Eddie, Bill) easy to understand?

easy/hard

1.7

3) Was it hard to remember the three characters?

easy/hard

2.3

4) Did the videos of the person using the three characters help you to

helped/no help

2.7

understand the characters?

5) Did you like doing the MEBi exercises?

not liked/liked

6.4

6) Were the calls with the research staff useful?

not useful/useful

6.4

7) Are the skills in MEBi useful to you in social situations?

not useful/useful

6.6

8) Has MEBi changed how you think in social situations?

no/yes

5.6

9) Has MEBi changed how you act in social situations?

no/yes

5.6

managing their day-to-day social interactions. In addition to quantitative feedback, participants were asked to comment on what they did and did not like about the MEBi intervention. Most of the negative comments criticized the functioning of the iPad app. For example, “Sometimes the screen froze up, but once I talked to [research staff] it got fixed and I could continue.” And “Well sometimes I had to press the buttons 20 times before the picture changed.” Positive responses included, “It helped me to understand people in social situations,” “I liked the videos [illustrating use of the Mary/Eddie/Bill heuristic],” “I enjoyed the exercises,” and “It helped my emotions; helped me understand my feelings.” The MEBi-Standard and MEBi-Self groups did not differ significantly in responses to any of the feedback items with the exception that the former gave significantly higher ratings on Item 8, “Has SCIT changed how you think in social situations?” (p < .05).

Posttest assessment of participants’ recall for the MEBi heuristic revealed that 15 of the 24 completers correctly named all three characters and correctly identified a characteristic emotion, behavior, or cognition of each. Of the remaining nine participants, six either named all characters but not a characteristic of all three, or vice-versa (e.g., “One is angry, one is happy and one is sad.”). Finally, none of the iPads was damaged or lost during this study.

Efficacy data are shown in Table 3. The MEBi-Standard group improved relative to the MEBi-Self group on the WRT self-referential bias (F = 5.405, p = .030) and accuracy scores (F = 5.66, p = .026), the SCSQ hostile attribu-tional bias score (F = 4.67, p = .042), and the RIM (F = 5.405, p = .030). Effect sizes for all significant interactions were in the small to moderate range. The groups did not differ on the SCSQ ToM or overconfidence bias score, or on the Hinting task. On in-session ratings of overconfidence (Figure 1), MEBi-Self participants showed statistically significant increases from treatment midpoint to end point (F = 8.42, p < .001), while the MEBi-Standard participants showed no change.

Table 3. MEBi-standard vs MEBi-self efficacy data.

Test

MEBi Standard (n = 10)

MEBi Self (n = 14)

Time x Group Effect size

Pre

Post

Pre

Post

Waiting room task

Self-referential bias0

.318 (0.26)

.172 (0.14)

.095 (0.12)

.123 (0.17)

.299

Theory of mind accuracy6

34.6 (3.5)

36.8 (3.3)

37.00 (4.0)

35.9 (6.7)

.205

SCSQ

Hostile attributional bias +

2.40 (0.84)

2.10 (0.74)

1.79 (0.80)

2.29 (0.73)

.175

Theory of mind accuracy

5.60 (1.71)

6.00 (1.49)

5.79 (1.19)

5.71 (0.99)

.021

Overconfidence bias

2.23 (1.33)

2.00 (1.55)

1.76 (1.75)

1.57 (1.45)

.000

Hinting task

10.65 (5.87)

11.80 (5.01)

11.14 (5.13)

12.21 (4.15)

.000

Relational interactivity measure0

15.60 (5.23)

19.30 (4.02)

20.07 (4.94)

18.29 (6.01)

.197

SCSQ = Social Cognition Screening Questionnaire.

Time x Group interaction effects: aF = 5.405; p = .030; bF = 5.66; p = .026; +F = 4.67; p = .042; cF = 5.405; p = .030.

Because the treatment groups were not randomized and differed in baseline performance on all four of the WRT and SCSQ subscales, post hoc analyses were performed to assess whether observed effects may be attributable to measurement error and regression to the mean. We could not discount this form of error in SCSQ performance. In contrast, both WRT results appear to reflect a differential treatment effect depending on baseline social cognitive ability. Namely, individuals who performed poorly at baseline in terms of accuracy and/or bias tended to improve on these measures if they received MEBi-Standard but worsened if they received MEBi-Self.

Discussion

We predicted that the month-long in-home tablet-based MEBi intervention would be feasible among outpatients with schizophrenia, and that the MEBi-Self version would yield greater improvements in social cognition than the MEBi-Standard version. These hypotheses were partially supported. Feasibility data strongly supported the ability to use in-home tablet-based training as a treatment modality in this population. However, in contrast to our hypothesis, the MEBi-Self group performed significantly worse after the intervention on several measures relative to the MEBi-Standard group. These findings are discussed in detail below.

Regarding feasibility, this is the first study to our knowledge to target social cognition in schizophrenia through in-home, tablet-based intervention. Feasibility data were uniformally positive, including high treatment adherence, low dropout (two of 24 participants), no lost or damaged tablets, and generally positive feedback on the intervention. An additional benefit of this treatment approach is the ability of the research team to track participants’ adherence in real time through finger-stroke data and the tablets’ wifi connectivity. This enabled staff to identify quickly participants who were falling behind in the training schedule and address this during phone check-ins. None of the participants expressed concern about being monitored in this way through the tablet computers, and several expressed gratitude for the responsiveness of the staff in troubleshooting technical issues and answering questions about the MEBi training content. Results from the posttreatment assessment of participants’ recall for the MEBi heuristic is in line with a previous group-based version of MEB (Roberts et al., 2012), indicating that participants learned the heuristic as effectively through tablet interface as through group interaction. This, and the feasibility data more broadly, support ongoing development of in-home, computer-based interventions for psychosis.

Efficacy data were somewhat surprising. Of the eight outcome measures, five showed a similar pattern in which MEBi-Standard participants performed worse at baseline, and then improved, whereas MEBi-Self participants performed better at baseline, and then got worse. Several factors support the validity of this observed pattern. First, the finding was present across domains, including self-referential bias, overconfidence bias in in-session finger-stroke responding, social cognitive accuracy and self-reported social adjustment. This finding is promising in that one goal of the study was to activate social cognitive bias, and we are aware of no other studies that have effectively activated social cognitive bias in schizophrenia through a computer interface. The fact that our self-referential manipulation did not merely increase self-referential bias, but appears to have had broader effects on overconfidence bias, diminished accuracy and diminished social engagement, is also interesting as it differs from previous findings which suggest the independence of bias domains in schizophrenia (Moritz et al., 2010).

On the other hand, the aim of MEBi-Self was not to render participants more dysfunctional, but to activate bias and then have participants neutralize the bias through the application of the MEB debiasing technique. The current evidence suggests that bias was activated but was not subsequently neutralized. The bias intervention may have overpowered the debiasing tool or, alternatively, participants may not have attempted to use the debiasing tool to neutralize biases once activated. This latter possibility could be because there was no component of the MEBi-Self intervention that explicitly prompted participants to apply the debiasing tool to feelings of self-referenti-ality and overconfidence in response to self-relevant stimuli. Alternatively, patients may not have become aware of their increased bias and so may not have thought to apply the tool. This interpretation is consistent with findings in experimental social psychology that show that people typically are not aware of bias activation, but if they are made aware, they can effortfully neutralize bias (Schwarz & Clore, 2007). Thus, these findings suggest the need to test a modified form of the MEBi-Self intervention that draws peoples’ attention to their increased bias and prompts them to use the debiasing technique.

Another interpretation of the negative MEBi-Self findings is that participants in this condition did not receive sufficient rehearsal with the MEBi skill. Although MEBi-Standard participants received 24 sessions of basic training with the tool, MEBi-Self participants received only 12 basic sessions before moving on to 12 sessions of self-relevant content.

In sum, the present study suggests that in-home, tablet-based intervention for schizophrenia is feasible and well tolerated. Further, we were able to teach the heuristic strategy through tablet training in a manner that led to a high rate of learning among participants, and we were able to use a self-referential manipulation to increase social cognitive bias in a subset of our sample. Key limitations are that this study is underpowered and group assignment was sequential rather than random. Future work is needed to replicate current findings in a larger, well-controlled trial, and to determine whether it is possible to neutralize activated biases through tablet interface. To that end, we are currently conducting a larger, randomized study in which MEBi includes an enhanced debiasing intervention.

Funding

This study was supported by a grant to the first author from the Hogg Foundation for Mental Health.

References

Beck, J. (2005). Cognitive therapy: Basics and beyond. New York, NY: Guilford.

Bentall, R. P., Corcoran, R., Howard, R., Blackwood, N., & Kinderman, P. (2001). Persecutory delusions: A review and theoretical integration. Clinical Psychology Review, 21(8), 11431192. doi:10.1016/s0272-7358(01)00106-4

Combs, D. R., Penn, D. L., Micher, M., & Waldheter, E. (2007). The Ambiguous Intentions Hostility Questionnaire (AIHQ): A new measure for evaluating hostile social-cognitive biases in paranoia. Cognitive Neuropsychiatry, 12(2), 128-143. doi:10.1080/1354680060 0787854

Corcoran, R., Mercer, G., & Frith, C. (1995). Schizophrenia, symptomatology and social inference: Investigating “theory of mind” in people with schizophrenia. Schizophrenia Research, 17, 5-13. doi:10.1016/0920-9964(95)00024-g

Couture, S. M., Penn, D. L., & Roberts, D. L. (2006). The functional significance of social cognition in schizophrenia: A review. Schizophrenia Bulletin, 32, S44-S63. doi:10.1093/ schbul/sbl029

Evans, S. L., Averbeck, B. B., & Furi, N. (2015). Jumping to conclusions in schizophrenia. Neuropsychiatric Disorders and Treatment, 11, 1615-1624.

Fett, A. J., Viechtbauer, W., Dominguez, M., Penn, D. L., van Os, J., & Krabbendam, L. (2011). The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: A meta-analysis. Neuroscience & Biobehavioral Reviews, 35, 573-588. doi:10.1016/j.neubiorev.2010.07.001

First, M. B., Gibbon, M., Spitzer, R. L., & Williams, J. B. W. (1996). Structured clinical interview for DSM-IV Axis I disorders. Patient edition. New York, NY: Biometrics Research.

Garety, P. A., & Freeman, D. (1999). Cognitive approaches to delusions: A critical review of theories and evidence. British Journal of Clinical Psychology, 38, 113-154. doi:10.1348/ 014466599162700

Granholm, E., McQuaid, J. R., Auslander, L. A., & McClure, F. S. (2004). Group cognitive-behavioral social skills training for older outpatients with chronic schizophrenia. Journal of Cognitive Psychotherapy, 18(3), 265-279. doi:10.1891/jcop.18.3.265.65652

Green, M. F., Penn, D. L., Bentall, R., Carpener, W. T., Gaebel, W., Gur, R. C., ... Heinssen, R. (2008). Social cognition in schizophrenia: An NIMH workshop on definitions, assessment, and research opportunities. Schizophrenia Bulletin, 34(6), 1211-1220. doi:10.1093/schbul/ sbm145

Green, M. J., & Phillips, M. L. (2004). Social threat perception and the evolution of paranoia. Neuroscience and Biobehavioral Reviews, 28, 333-342. doi:10.1016/j.neubiorev. 2004.03.006

Hogg, L., & Hall, J. (1992). Management of long-term impairments and challenging behavior. In M. Birchwood & N. Tarrier (Eds.), Innovations in the psychological management of schizophrenia: Assessment, treatment and services (pp. 171-203). London, England: Wiley.

Hooker, C., & Park, S. (2005). You must be looking at me: The nature of gaze perception in schizophrenia patients. Cognitive Neuropsychiatry, 10(5), 327-345. doi:10.1080/ 13546800444000083

Kapur, S., Mizrahi, R., & Li, M. (2005). From dopamine to salience to psychosis—Linking biology, pharmacology and phenomenology of psychosis. Schizophrenia Research, 79, 59-68. doi:10.1016/j.schres.2005.01.003

Kurtz, M. M., Seltzer, J. C., Shagan, D. S., Thime, W. R., & Wexler, B. E. (2007). Computer-assisted cognitive remediation in schizophrenia: What is the active ingredient? Schizophrenia Research, 89(1-3), 251-260. doi:10.1016/j.schres.2006.09.001

McKay, R., Langdon, R., & Coltheart, R. (2007). Jumping to delusions? Paranoia, probabilistic reasoning, and need for closure. Cognitive Neuropsychiatry, 12(4), 362-376. doi:10.1080/ 13546800701203769

Moritz, S., Veckenstedt, R, Hottenrott, B., Woodward, T. S., Randjbar, S., & Lincoln, T. M. (2010). Different sides of the same coin? Intercorrelations of cognitive biases in schizophrenia. Cognitive Neuropsychiatry, 15(4), 406-421. doi:10.1080/13546800903399993

Moritz, S., Woodward, T. S., Stevens, C., & Hauschildt, M. (2013). Metacognitive training for psychosis (MCT), 4th volume. Hamburg, Germany: VanHam Campus Press.

Overall, J. E., & Gorham, D. R. (1962). The Brief Psychiatric Rating Scale. Psychological Reports, 10, 799-812. doi:10.2466/pr0.1962.10.3.799

Peer, J. E., Rothmann, T. L., Penrod, R. D., Penn, D. L., & Spaulding, W. D. (2004). Social cognitive bias and neurocognitive deficit in paranoid symptoms: Evidence for an interaction effect and changes during treatment. Schizophrenia Research, 71, 463-471. doi:10.1016/j. schres.2004.03.016

Reitan, R. M., & Wolfson, D. (1985). The Halstead-Reitan neuropsychological test battery. Tucson, AZ: Neuropsychological Press.

Roberts, D. L., & Hoffman, R. (2011, April). Social deafferentation and psychosis. In A. Mishara (Chair) Bridging Clinic and Clinical Neuroscience: Loneliness, Social Anhedonia and Bonding in Schizophrenia Workshop conducted at 13th International Congress on Schizophrenia Research, Colorado Springs, CO.

Roberts, D. L., Kleinlein, P., & Stevens, B. J. (2012). An alternative to generating alternative interpretations in social cognitive therapy for psychosis. Behavioural & Cognitive Psychotherapy, 40, 491-495. doi:10.1017/s1352465812000082

Roberts, D. L., Penn, D. L., & Combs, D. R. (2015). Social Cognition and Interaction Training (SCIT): Treatment manual. New York, NY: Oxford University Press.

Rosenfeld, A. J., Lieberman, J. A., & Jarskog, F. (2011). Oxytocin, dopamine, and the amygdala: A neurofunctional model of social cognitive deficits in schizophrenia. Schizophrenia Bulletin, 37(5), 1077-1087. doi:10.1093/schbul/sbq015

Schwarz, N., & Clore, G. L. (2007). Feelings and phenomenal experiences. In E. T. Higgins & A. Kruglanski (Eds.), Social psychology. Handbook of basic principles (2nd ed., pp. 385-407). New York, NY: Guilford.

Winton-Brown, T. T., Fusar-Poli, P., Ungless, M. A., & Howes, O. D. (2014). Dopaminergic basis of salience dysregulation in psychosis. Trends in Neuroscience, 37(2), 85-94. doi:10.1016/j.tins.2013.11.003

Woodward, T. S., Moritz, S., Cuttler, C., & Whitman, J. C. (2006). The contribution of a cognitive bias against disconfirmatory evidence (BADE) to delusions in schizophrenia. Journal of Clinical and Experimental Neuropsychology, 28, 605-617. doi:10.1080/ 13803390590949511

Wykes, T., Reeder, C., Landau, S., Everitt, B., Knapp, M., Patel, A., & Romeo, R. (2007). Cognitive remediation therapy in schizophrenia: Randomised controlled trial. British Journal of Psychiatry, 190(5), 421-427. doi:10.1192/bjp.bp.106.026575

Web-Based Psychoeducational Intervention for Persons With Schizophrenia and Their Supporters: One-Year Outcomes

Armando J. Rotondi, Ph.D.

Carol M. Anderson, Ph.D.

Gretchen L. Haas, Ph.D.

Shaun M. Eack, Ph.D.

Michael B. Spring, Ph.D.

Rohan Ganguli, M.D.

Christina Newhill, Ph.D.

Jason Rosenstock, M.D.

Objective: This study examined the use of a uniquely designed Web site and home computers to deliver online multifamily psychoeducational therapy to persons with schizophrenia and their informal supports (family and friends). Web site usage and outcome benefits are reported. Methods: Thirty-one persons with schizophrenia or schizoaffective disorder and 24 support persons were randomly assigned to the online intervention (telehealth) or treatment as usual (usual care) condition. At three, six, and 12 months, interviewer-administered assessments were conducted with participants. Intention-to-treat analyses compared persons with schizophrenia in the two study conditions on severity of positive symptoms and knowledge of schizophrenia. Support persons in the two study conditions were compared on knowledge of schizophrenia. Each participant’s usage of the Web site was logged. Results: Persons with schizophrenia in the telehealth condition had a large and significant reduction in positive symptoms (p=.042, d=-.88) and a large and significant increase in knowledge of schizophrenia compared with their counterparts in the usual care condition. Support persons in the telehealth condition showed a large and significant increase in knowledge about prognosis compared with those in the usual care condition (p=.036, d=1.94). Persons with schizophrenia used the Web site to a much greater extent (pages viewed and time spent) than support persons. Conclusions: These findings suggest that online delivery of psychotherapeutic treatment and educational resources to consumers’ homes has considerable potential to improve consumer well-being and offers several advantages over standard clinic-based delivery models. (Psychiatric Services 61:1099-1105, 2010)


Dr. Rotondi is affiliated with the Department of Critical Care Medicine, Dr Eack and Dr Newhill are with the Department of Social Work, and Dr Spring is with the Department of Information Sciences, all at the University of Pittsburgh, 644 Scaife Hall, 3550 Terrace St., Pittsburgh, PA 15260 (e-mail: rotondi@pitt.edu). Dr Anderson, Dr Haas, Dr Ganguli, and Dr Rosenstock are with the Department of Psychiatry, University of Pittsburgh Medical Center Dr Rotondi and Dr Haas are also with the Veterans Integrated Service Network 4, Mental Illness Research, Education and Clinical Center, Veterans Affairs Pittsburgh Healthcare System.


Because treatments that have proved efficacious in academic settings over the past 30 years have rarely been adopted by community agencies (1,2), there is a need to develop ways to translate and disseminate useful programs to community settings. One such treatment is family psychoeducation (3-5). The results of more than 30 experimental trials have established family psychoeducation as an evidence-based practice (6,7) routinely recommended as the gold standard for care of persons and families coping with schizophrenia (1,8-11). However, results from the seminal Schizophrenia Patient Outcomes Research Team study showed that although 77% of those with schizophrenia live with or have ongoing contact with their families, only a small proportion have an outpatient claim for any family intervention (<1% Medicare and 7% Medicaid) (2). These data led us to design a randomized feasibility trial of an adapted telehealth family psy-choeducational intervention provided to users' homes to increase community dissemination.

Online interventions have the potential to overcome several barriers to dissemination and community care for persons with serious mental illness, including the limited number of persons with schizophrenia in some clinics, the high costs of training staff

and delivering evidence-based programs, the difficulty of monitoring fidelity, and the interpersonal and logistical difficulties involved with traveling to receive services. Telehealth services also allow individuals to take an active role in seeking information and support, to participate in group interactions, to learn specific skills, and to adjust participation as necessary. Thus the online method can facilitate the traditional treatment goals of symptom and burden reduction and facilitate the recovery goals of improved quality of life and personal goal attainment.

The strategies used in this “schizophrenia online access to resources” (SOAR) intervention were designed to provide key elements of family psychoeducation: empathic engagement of participants, education about the illness and treatments, a supportive safety net, and coping strategies. The interventions provided and the Web site's format were designed to be clear, accessible to persons with cognitive impairments (12), and visually nondistracting and low key to avoid an exacerbation of the illness-related vulnerabilities to stimulation that can trigger symptoms and relapse (8,13). Based on evidence suggesting the advantages of using a group model (14), we added supporter and peer discussion forums to SOAR.

SOAR teaches and engages participants in problem solving to decrease stress, meet personal needs, and achieve personal goals. Theoretical and empirical work indicates that meeting an individual's needs can reduce stress, promote better adaptation to illness-related difficulties, and improve outcomes (15), whereas when needs go unmet distress can increase and outcomes may worsen (16). In addition, through tutorials and self-help articles, the content on the SOAR site emphasizes promotion of self-efficacy, self-management, and problem solving (17,18). These themes were also emphasized in three therapy forums on the site, which encourage social support and social learning among participants (19).

Methods

This study was conducted from October 2003 to April 2005. A more detailed description of the methods and participants is available elsewhere (20).

Participant recruitment and selection criteria

Persons with schizophrenia or schizoaffective disorder were recruited from community mental health centers and inpatient units. Enrollment criteria for persons with schizophrenia were age 14 or older, diagnosis of schizophrenia or schizoaffective disorder (DSM-IV criteria), one or more psychiatric hospitalizations or emergency department visits within the previous two years, the ability to speak and read English, living in the community at the time of study entry, and absence of physical limitations that would preclude using a computer. Enrollment criteria for support persons were age 18 or older, ability to speak and read English, and absence of physical limitations that would preclude using a computer. [Additional enrollment details are provided in an appendix available as an online supplement to this article at ps.psychiatryonline.org.] Informed consent was obtained from participants, and the research protocol was approved by the University of Pittsburgh Institutional Review Board.

Participants

Thirty-one persons with schizophrenia or schizoaffective disorder and 24 support persons were randomly assigned to the telehealth or usual care condition.

Procedures

Participants assigned to the telehealth condition received dial-up Internet access and a computer as needed. Participants were in the study for up to one year (see online appendix for additional details).

SOAR Web site

Each intervention participant was given a unique login name and password to access SOAR. The SOAR software collected information on each participant's usage of the Web site, including the pages visited, time of day the site was visited, time spent on the site, and user identifier. (See the online appendix for greater detail about the content and design of the site.)

Content. Selection of the content domains and their creation are described elsewhere (12,20). The SOAR Web site provides the following components: three therapy forums, one for persons with schizophrenia only, one for support persons only, and one for both groups of users; a capability for asking questions of and receiving answers from the project team within 24-48 hours; a library of previously asked and answered questions; a library of educational reading materials; and a list of community activities, events, and resources.

Facilitation of therapy forums. Each forum was led and moderated by a therapist. In each forum the therapist emphasized discussions that focused on problem solving, alleviating stress, and interacting with peers to develop a supportive forum where members could work together to address problems (3,21).

Site design. SOAR was specifically designed to be accessible to persons with cognitive impairments, as detailed elsewhere (12). The design model is based on five key principles: use of explicit links and labels; a flat, one-page-deep hierarchy; a relatively high number of links on navigation pages; a single constant navigational toolbar; and minimal superfluous and distracting content, such as images.

Psychoeducation survival skills workshop

Before we installed computers in participants' homes and provided access to SOAR via a desktop icon, telehealth participants (persons with schizophrenia and their support persons) attended a joint, four-hour psychoeducation survival skills workshop. The course was modeled on a workshop developed by Dr. Anderson and her colleagues (3) and is described elsewhere (20).

Assessment instruments

Trained, nonblinded interviewers collected data from participants at study entry and at three, six, and 12 months postbaseline. (Additional details about assessment tools are provided in the online appendix.)

Scale for the Assessment of Positive Symptoms. The 34-item Scale for the Assessment of Positive Symptoms was used at all time points to assess for the presence and severity of positive symptoms associated with schizophrenia (22).

Knowledge About Schizophrenia Instrument. At baseline and at six months, the Knowledge About Schizophrenia Instrument was administered to assess participants' clinical knowledge about the illness (23).

Usage of the Web site. Server logs from October 2003 to April 2005 were analyzed to determine usage patterns and number of page views or hits. The time between accessing one page and viewing another was considered to be the amount of time that a page was available for viewing. For usage analyses, online time spent on educational activities included viewing reading materials, viewing the library of questions and answers, and submitting questions to be answered and viewing responses.

Data analytic plan

Intention-to-treat analyses were used to investigate the effects of the intervention versus usual care conditions on outcomes. The three-month data collected from one person with schizophrenia was deemed unreliable by the interviewer and was thus excluded from analyses. Mixed-effects random-intercept models characterized the rates of change over the course of the study, after analyses adjusted for baseline age, gender, and positive symptoms. These models used an autoregressive error structure most appropriate for longitudinal data (24) and used restricted maximum-likelihood estimate model parameters when data were missing (25).

The significance of treatment effects was examined by calculating two-tailed t tests of the beta weight for the mixed-model treatment x time interaction term, as is customary. Cohen's d was calculated from predicted means based on these models to estimate the magnitude of treatment effects on rates of changes to symptomatology (schizophrenia group) and knowledge (schizophrenia and support persons). In addition, within-group analyses of the associations between Web site usage and baseline symptomatology and knowledge outcomes were conducted within the intervention group using Pearson correlation coefficients to identify the correlates of the intervention. For symptomatology data collected at multiple follow-up periods, correlation coefficients were pooled across time to obtain an average estimate of the relations between symptomatology and amount of SOAR Web site usage. Positive symptom data were log-transformed because of substantial positive skew. No adjustments for multiple comparisons were made in this initial feasibility study of the intervention.

Results

Participants

Demographic characteristics of the study sample are presented in Table

Use of the Web-based intervention Persons with schizophrenia in the telehealth group. The total amount of time spent on the SOAR Web site by those in the schizophrenia group was 43,789 minutes (730 hours), which involved 47,630 page views. Over the course of the study, the average total time spent on the Web site per person was 46 hours, or 2,737±3,692 minutes (medi-an=971, range=162-11,796), with an average of 2,977±4,546 page views (median=1,170, range=189-14,120) per person. Usage of the main components of the Web site is presented in Table 2.

Support persons in the telehealth group. The total amount of time that support persons spent on the Web site over the period of observation was 10,901 minutes (182 hours) and involved 8,210 page views. The average total time spent on the Web site per person was 14 hours, or 839±1,125 minutes (median=372, range=30-4,021), with an average of 632±729 page views (median=253, range=19-2,498) per person. Support persons' usage of the main components of the Web site is presented in Table 2.

Engagement in treatment

Although treatment engagement involves a complex set of processes (26), an established operational defi-

Table 1

Demographic characteristics of persons with schizophrenia and their supporters who were randomly assigned to a telehealth intervention or to usual care

Persons with schizophrenia                             Support persons

Characteristic

Telehealth (N=16)

Usual care (N=15)

pa

Telehealth (N = 13)

Usual care (N=11)

pa

N

%

N

%

N

%

N

%

Age (mean±SD)

38±11

38±11

.859

47±14

53±13

.364

Male

6

38

4

27

.704

4

31

5

45

.675

Race

.652

.992

Caucasian

9

56

6

40

6

46

5

45

African American

6

38

8

53

6

46

5

45

Other

1

6

1

7

1

8

1

9

a From a t test or chi square test of differences between telehealth and usual care groups


Table 2

Individual usage of the main components of the schizophrenia online access to resources (SOAR) Web sitea

Users with schizophrenia (N=16)

Users supporting person with schizophrenia (N=12)

Page viewsb

Time on SOAR (minutes)

Page viewsb

Time online (minutes)

Site component

M SD

Mdn Range M   SD Mdn Range

M SD Mdn Range

M SD Mdn Range


Therapy forum

For persons with

schizophreniac

1,838

2,795

586

41-8,725

1,874

2,728

713

49-8,305

For all users

414

882

70

6-3,127

334

665

99

1-2,461

162

223

55

6-780

243

367

114

13-1,392

For support personsd

243

307

80

11-889

335

495

157

5-665

Educational materials

Questionse

113

217

42

3-886

124

198

53

3-732

58

65

46

1-198

90

106

58

5-315

Articles

69

59

51

11-207

163

172

86

19-579

52

55

32

7-169

127

173

45

8-568

a Covers period from October 1, 2003, to April 30, 2005

b Number of times pages on the SOAR Web site were viewed (page “hits”) by participants

c This forum was not available to support persons.

d This forum was not available to persons with schizophrenia.

e Included submitting questions to “Ask an Expert” and viewing the “Previous Questions and Answers” library.


nition of successful initial engagement (27-29) is return for treatment after the intake session. To explore engagement in the online treatment environment, we considered attendance at the psychoeducation survival skills workshop as the intake session and defined participants as initially engaged if they subsequently participated at least once in one of the therapy forums and in educational activities. All persons with schizophrenia participated in a therapy forum on at least 13 separate visits, and they used educational resources on at least four visits. These results indicate that 100% of the participants with schizophrenia were engaged in their treatment.

Of the support persons, only one did not fully engage (participating once in each forum but not in educational activities). Of the other 12 support persons in the telehealth condition, all participated in a therapy forum on at least 11 visits and viewed articles on at least five visits, indicating that 12 (92%) were successfully engaged in treatment.

OOngoing Web site usage Participants in the schizophrenia group had a total of 4,473 individual sessions or visits to the Web site, with a mean of 280±337 sessions per person (median=128, range=28-1,232). The persons with schizophrenia were active in the therapy forums during 3,200 sessions (72% of total sessions), with an average of 200±318 sessions per person (median=82, range=13-1,173). The average number of months in which each person was active in a therapy forum was 11±4 (me-dian=11, range=4-17), where they spent an average of 3.2 hours per month or 193±316 minutes (median= 64, range=14-1,193). Overall, these data indicate strong engagement of most participants in this part of the intervention.

Persons with schizophrenia participated in educational activities during 807 visits (18% of total visits), with an average 50±80 visits per participant (median=30, range=4-340). The average number of months in which each person viewed educational materials was 8±4 (median=8, range=3-16), spending an average of 32±26 minutes per month (median=24, range=7-88). These data suggest that online approaches to education, using materials specifically designed and written for this population, can be well received.

Support persons had a total of 972 sessions on the SOAR site, with an average of 75±59 sessions per person (median=57, range=1-207). They participated in a therapy forum during 539 visits (55% of total), with an average of 42±40 visits per person (medi-an=19, range=1-129). The average number of months in which each support person was active in a therapy forum was 9±6 (median=7, range=1-19), and each spent an average of 55±43 minutes on the site per month (median=41, range=9-161). Support persons participated in educational activities during 339 visits (35% of total), averaging 26±29 visits per person (median=189, range=0-92) over an average of 7±6 months (median=6, range=0-18). They spent on average 29±42 minutes (median=17, range= 0-160) during an active month. Support persons used the forum and educational components of SOAR frequently and over a sustained period, indicating that family and friends can become engaged by this approach to psychoeducation.

Outcomes

Analyses using mixed-effects random-intercept models indicated that compared with persons with schizophrenia who received usual care, their counterparts in the telehealth group had significant and large reductions (30) in positive symptoms during the treatment intervention (t=-2.06, df=97, p = .042, d=-.88) (Figure 1). Using the same analytical methods, we found in the telehealth group a significant and large improvement in knowledge about the diagnosis of schizophrenia (t=-2.34, df=24, p=.028, d=.88) (Figure 1). No effects were found on other domains of knowledge about schizophrenia. Analyses of the association between cumulative Web site usage and symptom severity indicated that individuals with more severe positive symptoms tended to spend more time on the SOAR site (r=.65, p=.005) and to access SOAR more frequently (r=.62, p=.009). Cumulative Web site usage was weakly associated with magnitude of positive symptom reduction (based on within-subject reduction in positive symptoms from baseline to 12 months), but the relationship was not statistically significant. Supporters in the telehealth group showed a large improvement in knowledge about prognosis (t=2.32, df=14, p=.036, d=1.94). No significant effects were observed on other domains of schizophrenia-related knowledge.

Discussion

The findings are remarkable in terms of the high level of engagement in online activities of persons with schizophrenia (100%) and supportive persons in their lives (92%) and the substantial usage of telehealth resources. Positive symptoms and knowledge about schizophrenia improved. In addition, higher Web site usage was associated with higher rates of positive symptoms, suggesting that those most in need of treatment sought and used a greater “dose” of the telehealth intervention. Perhaps surprising was the relatively higher level of participation of persons with schizophrenia compared with their supporters, a finding that may reflect a generationbased discomfort with technology or caused by the support group's composition, a highly diverse yet relatively small group. More than one member of the group of support persons suggested that the group be larger or more homogeneous to allow members to more easily bond over common experiences. The sustained involvement with the SOAR site suggests that the intervention has the potential to be a viable approach to providing therapies such as family psychoeducation to clients and their support network.

Although family psychoeducation is commonly associated with reductions in relapse rates and intimately involved in controlling positive symptom exacerbations, only some psychoeducation programs have shown a


reduction in positive symptoms (31-33), whereas many have not (34). SOAR's significant influence on positive symptoms is consistent with the intervention's goals and the details of its operation. An explanation may lie in the common topics for problem solving and discussion, which focused on managing positive symptoms, medications, and side effects, including talking to psychiatrists about these issues to minimize their interference with activities. Thus, although evaluation of possible mediating variables such as medication adherence, side effect management, and readiness to talk with the prescribing psychiatrist about these issues was beyond the scope of this study, it is possible that concerns about symptom-related problems were positively influenced by the forum discussions and answers from the Ask the Expert feature.

Web site usage is certainly an indicator of engagement and may be a sufficiently valid indicator for most users, but it likely falls short as a measure of ongoing engagement for all users. Exposure to treatment is paramount but also encompasses such qualities as valuing and being committed to treatment and participation in treatment activities, such as problem solving. Feedback from users indicated that a range of Web site usage may be observed even among those who believe in the effectiveness of the treatment and are engaged. Usage may be influenced by need, which can vary over time because of intervention success, degree of symptom control, medication difficulties, severity of stresses, changes in coping resources, and so on. It has also been noted that disengagement may be an indication of successful behavior change (35). From this perspective, persons with schizophrenia who are more functional, have better personal and social resources, and have fewer problems taxing their ability to cope may still be engaged and benefit from intermittent use rather than more frequent or continuous use (see examples in the online appendix).

Because psychiatric symptoms and the underlying cognitive deficits of schizophrenia can interfere with an individual's ability to engage in meaningful and appropriate face-to-face interaction, computer-based interaction offers significant advantages (36,37). The visual display of words on a monitor may help compensate for deficits in auditory processing, attention, and memory and may improve concentration and attention to the task. Moreover, interactions online versus in person may be less influenced by cognitive deficits and social impairments. Individuals who are highly sensitive to external stimulation or uncomfortable with direct interpersonal interaction can more actively manage interpersonal exposure by gauging and adjusting the intensity of their interactions. They can access the Web site in small “doses,” even at odd hours when the resources of community clinics are not available. Finally, the experience of being able to personally and safely manage personal use of the Web site may generate a sense of empowerment and self-efficacy.

The telehealth model provides agencies and therapists with several advantages usually unavailable in community clinics: the ability to monitor patient participation and increase direct services as needed, provide groups that minimize professional impact and emphasize peer-driven interaction and problem solving, and provide proven informational and support services to supportive families and friends in a destigmatizing way.

Despite these apparent advantages, there are limitations to this study, including the relatively small sample. Although small, the sample was relatively heterogeneous with regard to demographic and clinical features. Given that the use of Webbased technologies can involve complex information processing and require flexible, time-sensitive sensorymotor and integrative cognitive functions (38), the cognitive deficits associated with serious mental illness must be taken into account in the design of these technologies. Because there was no earlier research on technology design for persons with serious mental illness, we initiated such research in preparation for this study, identifying features that influence Web site usage by this population. On the basis of these findings, we created an appropriate design for the SOAR Web site (12), which runs counter to current design guidelines for the general public. The unique interface design of SOAR was not only acceptable to users, but it was also likely to have been a contributing factor in their ability to access the site effectively.

Conclusions

Clearly there is a need for cost-effective ways of successfully delivering evidence-based programs to the community. Telehealth programs may offer an ideal solution. Experts based in central locations can provide a sophisticated intervention to underserved individuals and families living in widely dispersed areas, eliminating the need for individual agencies to have the necessary startup funds or clinical expertise. These findings suggest that when telehealth services are specifically designed to accommodate the cognitive deficits of persons with schizophrenia, the services can and will be used. Online delivery of psychotherapeutic treatment to consumers' homes can improve consumer well-being and offers several advantages over standard clinic-based delivery models.

Acknowledgments and disclosures

This project was supported by grant R01 MH63484 from the National Institute of Mental Health.

Dr. Eack receives consulting fees from Abbott Laboratories. Dr. Ganguli is a consultant for Janssen Pharmaceuticals, Johnson & Johnson, and Eli Lilly and Company, and he works under a research grant funded by Bristol Myers-Squibb. The other authors report no competing interests.

References

Coming in December

This article was downloaded by: [McMaster University]

On: 28 December 2014, At: 11:49

Publisher: Taylor & Francis

Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK


Assistive Technology: The Official Journal of RESNA

Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uaty20

Ecological Assessments of Activities of Daily Living and Personal Experiences with Mobus, An Assistive Technology for Cognition: A Pilot Study in Schizophrenia

Juliette Sablier PhD a b , Emmanuel Stip MD MSc b , Pierre Jacquet PhD c , Sylvain Giroux PhD d , Helene Pigot PhD d , Mobus group e & Nicolas Franck MD PhD c a Neuroscience Cognitive Center CNRS UMR 5229 , University of Lyon & Le Vinatier Hospital , France b Fernand Seguin Research Centre , University of Montreal & Louis-H. Lafontaine Hospital , Canada

c Neuroscience Cognitive Center CNRS UMR 5229 , University of Lyon & Le Vinatier Hospital (Rehabilitation Center) , France

d DOMUS Laboratory , University of Sherbrooke , Quebec , Canada

e Mobus group is composed of: Programmers at DOMUS Laboratory of the University of Sherbrooke , Quebec , Canada , Bouchard, F., Marcotte, N., Viboud, J.P.; Psychiatrists at Louis-H. Lafontaine Hospital (Quebec, Canada): Bentaleb, L.A., Landry, P., Lipp, O., Tranulis, C., Villeneuve, M.; Ergotherapists at Louis-H. Lafontaine Hospital (Quebec, Canada): Cloutier, C., Lalancette, C., Prince, A.; Pharmacists at Louis-H. Lafontaine Hospital (Quebec, Canada): Vincent, P., Lum, M.; Social worker at Vinatier Hospital (Lyon, France): Berrube, M.C.; Beneficiaries attendant at Louis-H. Lafontaine Hospital (Quebec, Canada): Lucas, M.; Nurses and psychologist at Vinatier Hospital (Lyon, France): Boisset, G., Guida, M., Mazuire, J., Meylan, F., Meynier, J., Pelletier, G, Sportiello, S.; Students at University of Montreal (Quebec, Canada): Dore-Gauthier, V., Guevremont, C., Nadeau-Marcotte, F. Accepted author version posted online: 06 Apr 2012.Published online: 06 Jun 2012.

To cite this article: Juliette Sablier PhD , Emmanuel Stip MD MSc , Pierre Jacquet PhD , Sylvain Giroux PhD , Helene Pigot PhD , Mobus group & Nicolas Franck MD PhD (2012) Ecological Assessments of Activities of Daily Living and Personal Experiences with Mobus, An Assistive Technology for Cognition: A Pilot Study in Schizophrenia, Assistive Technology: The Official Journal of RESNA, 24:2, 67-77, DOI: 10.1080/10400435.2012.659324

To link to this article: http://dx.doi.org/10.1080/10400435.2012.659324

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the "Content") contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Downloaded by [McMaster University] at 11:49 28 December 2014


Assistive Technology®, 24'.6~I-~n, 2012

Copyright© 2012 RESNA

ISSN: 1040-0435 print/1949-3614 online

DOI: 10.1080/10400435.2012.659324

Taylor & Francis

< J Taylor & Francis Croup


Ecological Assessments of Activities of Daily Living and Personal Experiences with Mobus, An Assistive Technology for Cognition: A Pilot Study in Schizophrenia

Downloaded by [McMaster University] at 11:49 28 December 2014


Juliette Sablier, PhD,1 Emmanuel Stip, MD, MSc,Pierre Jacquet, PhD,3 Sylvain Giroux, PhD,4 Helene Pigot, PhD,4 Mobus group,5 and Nicolas Franck, MD, PhD3

1

Neuroscience Cognitive Center CNRS UMR 5229, University of Lyon & Le Vinatier Hospital, France; and Fernand Seguin Research Centre, University of Montreal & Louis-H. Lafontaine Hospital, Canada 2Fernand Seguin Research Centre, University of Montreal & Louis-H. Lafontaine Hospital, Canada 3Neuroscience Cognitive Center CNRS UMR 5229, University of Lyon & Le Vinatier Hospital (Rehabilitation Center), France 4DOMUS Laboratory, University of Sherbrooke, Quebec, Canada 5Mobus group is composed of: Programmers at DOMUS Laboratory of the University of Sherbrooke (Quebec, Canada): Bouchard, F., Marcotte, N., Viboud, J.P.; Psychiatrists at Louis-H. Lafontaine Hospital (Quebec, Canada): Bentaleb, L.A., Landry, P., Lipp, O., Tranulis, C., Villeneuve, M.; Ergotherapists at Louis-H. Lafontaine Hospital (Quebec, Canada): Cloutier, C., Lalancette, C., Prince, A.; Pharmacists at Louis-H. Lafontaine Hospital (Quebec, Canada): Vincent, P., Lum, M.; Social worker at Vinatier Hospital (Lyon, France): Berrube, M.C.; Beneficiaries attendant at Louis-H. Lafontaine Hospital (Quebec, Canada): Lucas, M.; Nurses and psychologist at Vinatier Hospital (Lyon, France): Boisset, G., Guida, M., Mazuire, J., Meylan, F., Meynier, J., Pelletier, G, Sportiello, S.; Students at University of Montreal (Quebec, Canada): Dore-Gauthier, V., Guevremont, C., Nadeau-Marcotte, F.


Address correspondence to Juliette Sablier, 67 Boulevard Pinel, 69675 Bron Cedex, France. E-mail: juliette.sablier@agim.eu


ABSTRACT Mobus is a cognitive orthotic designed for people with difficulties managing Activities of Daily Living (ADL), as encountered in schizophrenia. It provides a schedule manager as well as the possibility to report occurrences of symptomatic experiences. Receiving this information by Internet, caregivers can assist the patient rehabilitation process. Our aim was to explore the use and satisfaction of Mobus by people with schizophrenia. Nine outpatients tested Mobus for 6 weeks. Indicators of cognitive functioning and autonomy were measured with the CAmbridge Neuropsychological Tests Automated Battery (CANTAB) and the Independant Living Skills Scale (ILSS). On average, 42.6% of the planned ADL were validated and more than 1 symptom per week were reported. Mainly because of technical breakdown, more than 50% of the outpatients evaluated the Mobus satisfaction below 1.7/5, nevertheless 3 participants appreciated it greatly. Some enhancements were found on subscales of CANTAB and ILSS and some participants reported that they acquired planning skills by using Mobus. To ensure ease of use, refinements are needed from rehabilitation and technical approaches, especially to personalize the device. Discussions on ethical and methodological issues lead to an improved version of Mobus that will be tested with a larger sample size.

KEYWORDS activities of daily living, assistive technology for cognition, cognitive remediation, executive functioning, MOBUS, schizophrenia, smart phone

INTRODUCTION

Schizophrenia is defined as a chronic psychosis that affects approximately 1% of the world’s adults. It is characterized by positive (hallucinations, delusions, disorganized behavior . . .) and negative symptoms (social avoidance, blunted affect, apragmatism . . .). Antipsychotics help to control these symptoms, but

Downloaded by [McMaster University] at 11:49 28 December 2014


cognitive impairments remain by most of schizophrenia people, and have functional impacts (Bowie & Harvey, 2005; Green, 1996). In particular, deficits in memory and executive functions lead to difficulties in planning Activities of Daily Living (ADL) (Chambon et al., 2008; Norman et al., 1999). Most of individuals with schizophrenia are indeed unable to assume social responsibilities such as finding a job, establishing interpersonal relationships, or living on their own. These difficulties represent barriers to autonomy and in performing social roles.

The evolution of new technologies contributes to provide remote services for patients as well as for caregivers by developing interactive assistance systems. Assistive Technology for Cognition (ATC), also referred as a “cognitive orthotic” or “prosthetic” (Cole, 1999; LoPresti, Mihailidis, & Kirsch, 2004) can be considered as a “crutch for the mind.” It has the advantage of being highly customizable, so that one can directly assist each patient in performing his ADL (LoPresti et al., 2004; Sablier, Stip, & Franck, 2009). Furthermore, these kinds of tools have positive social impacts, as they represent modernity, and are commonly perceived as making life more simple and enjoyable. With the advance of telecommunication, telecare systems offer an opportunity in assisting people anywhere at any time, providing ecological strategies of treatment. For example, Neuropage is a device aimed at assisting people with memory impairments to remember their ADL (Wilson, Emslie, Quirk, & Evans, 2001; Wilson, Evans, Emslie, & Malinek, 1997). It is a pager, for example, a little device composed of a screen where one line of text can be displayed, and one on/off button. Its use is extremely simplified in order to minimize the need of memory skills. The tool is linked to a paging company which sends messages automatically. These messages are chosen beforehand by the user (patient with memory deficits), in accordance with his/her caregiver. The patient is also an actor of his/her own treatment, and he/she takes decisions about the planning of his/her daily life. A ringtone alerts the patient when an activity has to be achieved. The use of Neuropage improved significantly the memory skills of more than 80% of the patients who achieved the experiment (Wilson et al., 2001). These improvements remained at least seven weeks after the end of use. This tool is also hopeful for people with memory impairments due to various pathologies. Nevertheless, its use is too simple for people with schizophrenia, who have more preserved memory skills than participants of the Wilson’s studies. As a matter of fact, it is important to propose tools adapted to the skills of targeted users, for example, not too simple, in order to motivate people to use them, but not too complex, in order to avoid discouragement. For example, the Planning and Execution Assistant and Training (PEAT), developed by Levinson (1997), offers the advantage of being very individualized. This system is able to change its plans in case of unexpected event. It creates also the best plan and provides visual and auditive interface for assisting the user. However, this user-interface is too sophisticated and complex for being used by people with cognitive impairments such as schizophrenia (Giroux et al., 2008). The challenge is also to balance easiness and complexity in order to develop ATC according to the needs and skills of targeted users. Granholm, Loh, and Swendsen asked 54 schizophrenia or schizoaffective people to test the Computerized Ecological Momentary Assessment (EMAc) (Granholm, Loh, & Swendsen, 2009). EMAc provides ecological measures of behavior, symptoms, as well as emotion. Patients were asked to fulfill a questionnaire on a Personal Digital Assistant (PDA) four times a day during one week. Recorded data was linked to scores at standardized assessment scales of daily functioning. Through this study, authors showed that EMAc was a valid and feasible approach for collecting ecological data about schizophrenia people. Finally, since 2003, the Research Laboratory on Domotics and Mobile computing of the University of Sherbrooke, Quebec, Canada (DOMUS) has been developing systems on PDAs for people with cognitive impairments, which eventually lead to the birth of Mobus (Giroux et al., 2008). This software supplies cognitive assistance and telemonitoring for ADL, as well as tools to gather ecological medical data through two connected sub-applications implemented in PDA: one for the patients, the other for the caregivers. On the one hand, the patients can consult their personalized list of ADL, and validate those which they actually realize. On another hand, they have the possibility to notify the caregivers when they feel a symptom, and to precise at which intensity. The caregivers can verify whether the patients validated their ADL, and they can consult information about intensity, frequency and time of apparition of the symptoms signaled by their patients. It is important

Downloaded by [McMaster University] at 11:49 28 December 2014


to note that the lists of ADL and symptoms are personalized according to the wish of each patient. A more detailed description of Mobus is available below (in the “material” paragraph) and in Giroux et al. (2008).

To sum up,

A conviviality study was conducted with the first version of Mobus. A conviviality study must satisfy technological and usability criteria. The usability criteria regroup efficacy, efficiency, and satisfaction criteria. It is also conducted to answer to the following questions:

Our conviviality study revealed that schizophrenic people were able to use Mobus. Furthermore, the application was appreciated by both patients and caregivers (Sablier et al., 2007). This first study allowed us to improve the application, which was tested through the present pilot experiment.

HYPOTHESES

More than 50% of ADL validation would reflect that patients used Mobus regularly, as well as signaling their symptoms at least one time per week. As no previous studies exist to provide reference rates, we chose values that were clinically consistent. Furthermore, we investigated the subjective appreciation of the device by the patients through a questionnaire for which the maximum score was five (see “Variables and Analysis” section below for a description of this appreciation questionnaire). We expected that the majority of the participants would attribute a score higher than 2.5, which corresponds to an appreciation between “appreciated a bit” and “neutral position.” Finally, we expected that the use of Mobus would enhance the skills on neuropsychological tests.

OBJECTIVE

Our aim was to explore the manner of usage and the appreciation, by schizophrenia people, of Mobus. The goal of using Mobus is to improve cognitive function as well as to treat impairments. As a matter of fact, we suppose that this external aid can compensate for cognitive handicap, as well as rehabilitate it. It means that the ATC first replace the cognitive function. Then, after a period of use, skills are developed and trained. Finally, the ATC can have longterm impact and induce stable improvements (Wilson et al., 2001). We also explored if the use of Mobus improved memory and executive functioning as well as autonomy of people with schizophrenia.

METHODOLOGY

Ethic

This research received ethic approval from the French General Management of Health (protocol 06 053).

Participants

Inclusion criteria were: diagnosis of schizophrenia according to DSM-IV criteria, age above 18, and treatment stable since at least 2 weeks. Fourteen individuals signed the consent form. Nine patients participated until the end of the experiment. See

TABLE 1 Characteristics, participation duration and locations of patients involved (N = 9)


Medication Name


Downloaded by [McMaster University] at 11:49 28 December 2014


Sex

Age

Subtype of schizophrenia

Disease duration

/ dose (mg) / dosage regimen

Locations

Participation duration

P1

F

30

Undifferentiated

13

Clozapine 300 mg

Supervised residence

5 months and

BID

+ CPTTAa

10 days

P2

F

39

Paranoid

15

Aripiprazole 15 mg

Parents' home + DHb

5 months and

BID

15 days

P3

F

44

Residual

14

Risperidone Consta

Parents' home +

6 months

LP 100 mg

CPTTAa

P4

M

32

Paranoid

9

Olanzapine 15 mg

Mother's home +

6 months

DHb

P5

F

44

Undifferentiated

?

Olanzapine 5 mg

Parents' home +

2 months and

Fluoxetine 20 mg

CPTTAa

20 days

P6

M

33

Undifferentiated

1

Amisulpride 400 mg

Parents' home

6 months

P7

M

25

Paranoid

1

Aripirazole 20 mg

Own home

2 months and

7 days

P8

F

33

Schizo-affective

7

Risperidone Consta

Supervised residence

2 months and

disorder

LP 50 mg

20 days

P9

F

52

Paranoid

7

Risperidone 2 mg BID

Hospital

2 months and

8 days

a

CPTTA =

Center for Part-Time and Therapeutic

Activities, bDH =

Day Hospital.

TABLE 2 Characteristics, participation duration and locations of four patients who were excluded (N = 4) and one who abandoned


Sex

Age

Subtype of schizophrenia

Disease duration

Medication

Steps completed

Locations

Exclusion (X) Abandon (A)

Reasons of exclusion or abandon

M

36

Undifferentiated

13

Aripiprazole

15 mg

T1, BL

Own home + CPTTAa

X

Feeling of persecution

M

30

Disorganized

10

Aripiprazole

15 mg

T1, BL, T2, SP

Mother's home DHb

X

Delirium further to admission to a residence

M

28

Paranoid

3

Risperidone

8 mg

T1

Mother's home

X

Did not want to complete his BL-questionnaire, mother hostile to the experiment

M

22

Paranoid

8

Haloperidol

20 mg

T1, BL, T2, SP

Supervised residence

X

Relapse

F

35

Disorganized

1

Aripiprazole

15 mg

T1, BL, T2

Mother's home CPTTAa

A

Fear of technology, mother hostile

to the experiment


aCPTTA = Center for Part-Time and Therapeutic Activities; bDH = Day Hospital.


Table 1 for characteristics of each participant. Reasons of exclusion from the study are presented in Table 2. Furthermore, this study involved nurses and a neuropsychologist from the Vinatier psychiatric hospital, France. The experimentation took place in the Center for Part Time and Therapeutic Activities (CPTTA),


and in a Day Hospital (DH). Patients from DH and CPTTA are supposed to be able to live on their own. The living and health-care locations of each patient encountered are described in Tables 1 and 2. To sum up, this study involved patients from various locations, with various degrees of autonomy. Fortunately, the


Downloaded by [McMaster University] at 11:49 28 December 2014


material presented here is highly customizable and can be adjusted to suit each individual.

Material

Mobus provides two connected sub-applications implemented in PDA: one for the patients, the other for the caregivers (Figure 1).

Patient Application

Patients can either consult their list of ADL or notify the caregivers about a symptom, simply by clicking on the words “act.” or “sympt.” displayed on the touch-screen (Figure 1a). During an interview, the patient is said to express which ADL and symptoms he would like to record. These lists are therefore totally individualized. A color code notifies when the activity has to be performed (Figure 2a). As soon as the patient has accomplished the current activity, he must click on the corresponding line. The click time is also recorded on a server and the ADL validation appears on the caregiver’s PDA. The user has also the possibility to record what symptom is being experienced and how intense it is (Figures 2c, 2d, 2e). This is recorded, and can be consulted by the caregiver on the server.

Caregiver Application

The caregivers can create, modify, and delete the occurrence, date, and time of ADL, as well as the list of symptoms of each patient. The patients first plan their activities on a paper, and discuss with the caregiver about the symptoms that disturb them the most. Then, the caregivers record these data from their PDA. The steps needed for programming the schedule and symptoms on PDAs are therefore not realized by the patients, as it would have represented a complex and useless cognitive demand. When the patients have difficulties to determine which ADL should be recorded in their PDA, the caregivers are here to help them. It is then possible to remotely verify whether the ADL have been validated or not (Figure 2b). A complete technical description of the functioning of Mobus is available in Giroux et al. (2008).

In total, Mobus is not only a pager which provides a recall of activity that has to be done. First, Mobus provides a list of ADL organized in time. Then, a color code informs the user if its time to do the task (yellow), if its too late (red), or if the activity has to be done in the future (grey). Finally, patients must make an effort for organizing their activities which are then registered in the device by caregivers. These features are aimed at developing mainly planning skills.

Variables and Analysis

“ADL recall” is the name of the function displayed by the software, whereas “ADL validation” is the variable measured to quantify how much patients used the “ADL recall” function. In the same way, “symptom notification” is the appellation of the function


Downloaded by [McMaster University] at 11:49 28 December 2014



provided by Mobus, whereas “signaling symptoms” is the corresponding outcome.

ADL Validation

The rate of use of the ADL recall function was calculated by multiplying the number of ADL actually validated by the patient by 100, and dividing the result by the total number of activities initially planned. A rate of 100% corresponds to a perfect use, whereas a rate of 0% means that the patient never used Mobus to validate his activities.

Reporting Symptoms

The mean of use of the symptom notification per week was obtained by reporting the total number of symptoms signaled on the total number of days of use, then multiplied by 7. We also calculated the mean percentage of usage of this function among the whole duration of use of the device, in order to compare it to the rate of use of ADL recall. A rate of 100% corresponds to a mean of one use per day, so the rate could be above 100% if patients signaled more than an average of one symptom per day.

Appreciation Questionnaire

The appreciation of Mobus was obtained with a questionnaire filled by the participant at the end of the experiment. The user was asked to answer on a 6-point Likert scale (from 0 = “not at all” to 5 = “very much”) to 8 questions such as: “How much did you appreciate Mobus?” or “According to you, to what extent did Mobus improve your autonomy?” As each patient could answer to a different amount of questions whether he lived in a supervised residence or according to the functionalities he preferentially used, we calculated the mean score of appreciation for each patient. The best appreciation would also be represented by a score of five. We hoped that the majority of the patients would appreciate the device and give a score above 2.5 (the middle value). The repartition of the scores through our sample was observed with the calculation of the mean and the three quartiles (Q1Q2, Q 3).

Downloaded by [McMaster University] at 11:49 28 December 2014


Social and Neuropsychological Assessments

The French translation of the Independent Living Skills Scale (ILSS) (Cyr, Toupin, Lesage, & Valiquette, 1994) was used to assess the patients’ autonomy. Attention, planning skills and episodic memory were assessed with subtests of the CAmbridge Neuropsychological Test Automated Battery (CANTAB). The Paired Associate Learning (PAL) was used to assess episodic memory and learning skills (Levaux et al., 2007; Strip, Lecardeur & Ali-Sephery, 2008), whereas the Stocking of Cambridge (SOC) was administered to evaluate planning and strategy (Green et al., 2009). The SOC measures the time taken by the participants to plan their movements, as well as the number of movements realized to accomplish the task. These variables are measured according to the number of maximum movements needed to reach the goal of the task. The PAL consists of one or more presentations of a set of stimuli and corresponding attempts to correctly recall the location of each. The software detects whether the location that the subject chose was correct, and elicits a score of errors according to the number of shapes. Finally, the participants’ symptomatology, their quality of life and self-esteem were assessed.

Such an extensive measurement battery was chosen for this small sample study in order to obtain indicators of tendency for a next study with more patients and a control group.

Statistics

Paired sample t-tests were conducted in order to compare mean scores before and after the use of Mobus. Fisher’s correlations were calculated in order to investigate the link between the use of Mobus and scores on scales.

Procedure

Interviews were organized between the experimenter, the patient, and a caregiver before and after the use of Mobus. Participants were informed about the progress and goals of the experiment and signed consent forms. Mobus was presented to the patient and each function was explained and tested until the patient managed to use it without any help. Patients used their PDAs, specially customized according to their needs, during six weeks. Social and neuropsychological assessments (ILSS and CANTAB) were collected before and after the use of Mobus, for example, during each interview.

RESULTS

The rates of use of both functionalities provided by Mobus (ADL recall and symptom notification), as well as the mean scores of appreciation by the each patient, are presented in Figure 3.

ADL Validation

Patients validated a mean of 42.61% of activities. The majority (38%) were validated during the planned periods, whereas 31% and 27% were validated too soon or too late, respectively. A third of the patients used Mobus for validating more than 82% of their activities (P2, P4, and P6).

Symptom Notification

Patients signaled a mean of 1.01 symptoms per week. In total, they used this function with a mean of 14.44% of the total use-time. Two patients used it more than average (P1 and P9). Four patients notified only one symptom, on one occasion. Only one patient never used the function (P4). Sixty-two percent of the symptoms were signaled after noon.

Comparison of Both Functions

As it can be seen on Figure 3, the highest users of “ADL recall” (P2, P4, and P6) were not the same people as the highest users of “symptom notification” (P1 and P9). Furthermore, as observed in Figure 3, the majority of participants (P1, P6, P5, P2, P7, and P4) used more the “ADL recall” than the “symptom notification” function.

Appreciation

In Figure 3, data are ranged by increasing score of appreciation: from the participant who appreciated Mobus the less (P8 gave a mean score of 0.4) to the person who appreciated Mobus the best (P4 gave the


Downloaded by [McMaster University] at 11:49 28 December 2014


maximum score). The dotted line in Figure 3 delimits the scores above and under the expected value (2.5). Among our sample, the repartition of the scores was (Mean, Q1, Q2, Q3) = (2.27, 1.0, 1.7, 3.8). No significant correlation was found between the scores of appreciation and, neither the rates of ADL validations, nor the mean notification of symptoms per week.

Social and Neuropsychological Assessments

The results of ILSS and subtests of CANTAB scores, as well as other neuropsychological assessments, are presented in Table 3.

Significant enhancement between Baseline and Endpoint was found for the “Food” subscale of the ILSS (t(8) = 2.32, p = .049). The “Food” subscale in ILSS measures the ability of cooking meals, managing grocery, eating, and conserving healthy food. An increase in this score means that the individual becomes more autonomous related to meals and food.

Furthermore, a significant decrease in the mean subsequent thinking time for the SOC subtest of the CANTAB appeared (t(8) = 3.67, p = .006), which means that performance on this test was enhanced. The SOC subtest of the CANTAB assesses planning and strategy skills. There were no significant improvements on other measures.

DISCUSSION Promising Results

This pilot study was aimed at verifying whether schizophrenia people would be able to use Mobus regularly, by validating more than 50% of their ADL and by signaling at least one symptom per week. Actually, the patients validated a mean of 1.01 symptoms per week, but validated a mean of 42.61% of their ADL. Nevertheless, a third of the patients used Mobus for validating more than 82% of their activities. Furthermore, when they did use Mobus, most of the participants validated their ADL during the planned periods. This is a promising indication of potential effective use of Mobus in the future. On another hand, the fact that symptoms seem to be experienced mainly after noon could be taken into account in the treatment by caregivers. Interestingly, the highest users of “ADL recall” (P2, P4, and P6) were not the same people as the highest users of “symptom

Downloaded by [McMaster University] at 11:49 28 December 2014


TABLE 3 Clinical, psychosocial functioning, and cognitive scores (p values are indicated for significant f-tests)

N = 9

Baseline mean (SD)

Endpoint mean (SD)

P

PANSSa

Positive scale

14.11 (6.03)

15.00 (5.61)

Negative scale

13.78 (3.15)

14.22 (5.29)

General scale

32.56 (5.53)

32.56 (5.92)

ILSSb

Total score

0.83 (0.18)

0.85 (0.18)

Personal hygiene

0.83 (0.18)

0.85 (0.18)

Appearance and clothes

0.78 (0.21)

0.81 (0.21)

Cleaning

0.70 (0.29)

0.81 (0.27)

Food*

0.70 (0.21)

0.81 (0.28)

.049

Health

0.88 (0.20)

0.94 (0.17)

Money (N = 8)

0.62 (0.27)

0.72 (0.29)

Leisure

0.65 (0.18)

0.67 (0.20)

Quality of life

15.31 (5.98)

16.33 (5.45)

Self-esteem

32.67 (3.08)

30.89 (4.28)

CANTABc

MOT mean error

8.84 (3.54)

8.20 (2.06)

MOT mean latency*

840.99 (41.01)

941.66 (159.03)

.031

PAL total errors (adjusted)

28.60 (7.46)

24.33 (23.69)

PAL total errors (6 shapes, adjusted)

6.80 (8.69)

3.44 (4.69)

SOC mean initial thinking time (5 moves)

9,824.90 (5383.27)

5,283.47 (2882.67)

SOC mean subsequent thinking time (5 moves)*

1,521.75 (2000.41)

604.47 (641.99)

.006

SOC problems solved in minimum moves

6.80 (2.39)

6.78 (1.30)

SWM between errors

34.60 (20.51)

31.44(16.71)

SWM strategy

33.80 (6.63)

33.33 (6.06)

aPositive and Negative Syndrome Scale. bIndependent Living Skills Scale. cCAmbridge Neuropsychological Test Automated Battery. Results are significant.


notification” (P1 and P9). Patients may find it difficult to use both functions. However, this explanation does not correspond to the reports of the patients themselves, who found the application easy to use. Furthermore, the majority of participants used more the “ADL recall” than the “symptom notification” function. Participants could have been more motivated to improve their autonomy than to report their symptoms, which focus on negative aspects of the disease. Knowing that people with schizophrenia have difficulties recognizing their illness, we changed the name of this function in the next study, in order to motivate more patients to use it. The name “Experiences” would be better, as it would avoid focusing on symptoms, which is far less stigmatizing and closer to modern approaches to rehabilitation centered to recovery of people. On another hand, the participants experiencing more frequent symptom attacks may have decreased attention on the tasks they had to do. Finally, we expected that the use of Mobus would enhance the skills on neuropsychological tests.

The significant enhancement on the score at the “Food” subscale of the ILSS could mean that, at the end of the experiment, patients performed better at tasks related to food. On another hand, significant decrease was found on the mean subsequent thinking time of the SOC. This could reflect that our procedure might have led to an improvement in executive functioning, as we expected. Nevertheless, these results must be replicated with more subjects and a control group.

Complementary Hopeful Observations

Some patients had few activities (social avoidance, apragmatism): planning ADL with an entertaining tool could have encouraged them to plan new occupations.

Downloaded by [McMaster University] at 11:49 28 December 2014


For example, P2 took steps to register for a bicycle course, P7 participated in a soccer course, and P3 spent more time in her own apartment (gradually leaving parents’ home). It is important to add here the importance of occupational rehabilitation work in parallel with Mobus. As a matter of fact, the caregivers supported the mobilization, the reintegration of significant activities and the development of an occupational schedule rewarding for the person. Overall, the PDA personalization was very important, because of the various degree of autonomy and occupation of patients. Finally, some of them reported that they used “ADL recall” at the beginning, but after several days, they began to habituate to the repeated activities and remembered them without reminders. This encouraging result represents the potential that some people will gain in their ability to learn and organize their activities through use of Mobus.

Limits

Through Mobus, patients are supposed to be permanently connected to their caregivers. This is aimed at avoiding isolation, and providing a reassuring environment to the patient. Nevertheless, some schizophrenia people can experience paranoid symptoms (which belong to positive symptomatology). When patients strongly presented this symptom, we observed that it was difficult, indeed impossible, to ask them to use Mobus. As a matter of fact, these patients fear more than others from being observed. For them, Mobus can represent the threatening eye of “big brother,” instead of the friendly hand of the caregiver. On another hand, the fact that patients validate their ADL does not mean that they actually did them. Nevertheless, it would be unethical to watch them, as well as to force them to complete their activities.

Mainly, the PDAs experienced connection breakdown. Consequently, some tasks were not updated, and sometimes, participants could not validate an activity, because they did not see it on their diary. Thus, the device was not constantly functional, what could explain the weak rate of ADL validation. In fact, some participants may have used the “symptom notification,” which did not encounter any problems, rather than the “ADL recall” which did not update. On another hand, some patients used the “ADL recall” function despite the technical problems and were also less motivated to use the “symptom notification.”

Furthermore, these technical problems could explain the weak scores on the appreciation questionnaire. As a matter of fact, more than the half of our sample attributed a score under 1.7 to Mobus, and only 3 participants gave a score higher than 2.5, which was the expected value. Among these persons, P4 was a high user of “ADL recall” and P9 was a high user of “symptom notification.” Nevertheless, it is worth noticing that, for other patients, the subjective appreciation of the device seems to be in contradiction with its use. For example, P1, P2, and P6 had a weak appreciation of Mobus whereas they noticeably used it. This could explain the lack of correlation between these amounts of use and the appreciation of Mobus. Thus, if the connection problem was the core factor of weak appreciation, it did not prevent patients from using the device. We also can hopefully imagine that, with a good functioning of Mobus, most of patients would use it, but also appreciate it. As a matter of fact, the results of P4 show that it is possible, with 100% of validated ADL and a total appreciation of the device.

Furthermore, the connection breakdowns could have biased neuropsychological measures. This bias, added to the small sample size and large intersubject variability, imposes to consider statistical analyses with caution. These data represent also indicators of tendency, and should be investigated again with a new version of Mobus and a control group.

Finally, the choice of the assessments can be discussed. As a matter of fact, the ILSS is more focused on daily living skills than on occupational schedule. An occupational questionnaire, such as the University of California, San Diego (UCSD) Performance-Based Skills Assessment (Patterson et al., 2001), for example, could be envisaged for future studies. Furthermore, the appreciation questionnaire has not been validated, thus the results can be contested. In the future, Quebec User Evaluation of Satisfaction with assistive Technology (QUEST) (Demers, Weiss-Lambrou, & Ska, 2002) could be used.

Solution

To avoid the feeling of being watched, and to improve the feeling of managing one’s own schedule, a function could be added, which would allow the person to choose when he/she wants to be logged and which data he/she agrees to share. The use of such functionality would depend on the ability and the wish of choosing, which themselves depend on the symptoms intensity and on the cognitive skills of each patient. As a matter of fact, this would make the device more complex (good cognitive skills needed), and give new responsibility to the user, which is not necessarily a wish from each patient. Thus, this possibility, which would correspond to the model of recovery, should be another personalized functionality according to the patient’s profile, like “ADL recall” and “symptoms notification.”

Concerning technical problems, programmers detected that new connection problems arose from mismatches in the code. Thus, they reshaped the entire underlying program which is now much more stable and connection problems have been repaired.

Downloaded by [McMaster University] at 11:49 28 December 2014


Future Directions

It is important to use Mobus in a real context of rehabilitation where the reintegration of significant activities and the development of an occupational schedule are part of the rehabilitation plan. We are currently testing the enhanced version of Mobus in a randomized controlled study involving two groups of people with schizophrenia. Furthermore, Mobus could be tested by people with cognitive impairments due to other neuropsychological disorder. As a matter of fact, it could help individuals with brain traumatic injuries, as well as neurodegenerative or neurodevel-opmental diseases. Finally, the effect of the age could be investigated, as younger patients could be thought to be more familiar with new technologies than older people.

CONCLUSION

Mobus provided ecological data that could be valuable for clinical follow-up of schizophrenia people. This pilot study was useful to improve Mobus as well as to provide clues for developing new technologies for cognitive rehabilitation. Overall, this study highlights the importance of an integrated approach that links together complementary therapeutic modalities such as cognitive remediation (Demily & Franck, 2008) and social rehabilitation. The use of Mobus will be optimal in this context of integrated approaches.

REFERENCES

Bowie, C. R., & Harvey, P. D. (2005). Cognition in schizophrenia: Impairments, determinants, and functional importance. Psychiatric Clinics of North America, 28(3), 613-633.

Chambon, V., Franck, N., Koechlin, E., Fakra, E., Ciuperca, G., Azorin, J. M., et al. (2008). The architecture of cognitive control in schizophrenia. Brain, 131(4), 962-970.

Cole, E. (1999). Cognitive prosthetics: An overview to the method of treatment. NeuroRehabilitation, 12, 39-51.

Cyr, M., Toupin, J., Lesage, A. D., & Valiquette, C. A. (1994). Assessment of independent living skills for psychotic patients: Further validity and reliability. Journal of Nervous and Mental Disease, 182(2), 91-97.

Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002). Enhancing independent time-management skills of individuals with mental retardation using a Palmtop personal computer. Mental Retard, 40(5), 358-365.

Demers, L., Weiss-Lambrou, R., & Ska, B. (2002). The Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0): An overview and recent progress. Technology and Disability, 14(3), 101-105.

Demily, C., & Franck, N. (2008). Cognitive remediation: a promising tool for the treatment of schizophrenia. Expert Review of Neurotherapeutics, 8(7), 1029-1036.

Giroux, S., Pigot, H., Paccoud, B., Pache, D., Sablier, J., & Stip, E. (2008). Enhancing a mobile cognitive orthotic: A user-centered design approach. International Journal of ARM, 9(1), 1-12.

Granholm, E., Loh, C., & Swendsen, J. (2009). Feasability and validity of computerised ecological momentary assessment in schizophrenia. Schizophrenia Bulletin, 34(3), 507-514.

Green, M. F. (1996). What are the functional consequences of neu-rocognitive deficits in schizophrenia? American Journal of Psychiatry, 153(3), 321-330.

Green, C. R., Mihic, A. M., Nikkel, S. M., Stade, S. C., Rasmussen, C., Munoz, D. P., & Reynolds, J. N. (2009). Executive function deficits in children with fetal alcohol spectrum disorders (FASD) measured using the Cambridge Neuropsychological Tests Automated Battery (CANTAB). Journal of Child Psychology and Psychiatry, 50(6), 688-697.

Levaux, M. N., Potvin, S., Sepehry, A. A., Sablier, J., Mendrek, A., & Stip, E. (2007). Computerized assessment of cognition in schizophrenia: promises and pitfalls of CANTAB. European Psychiatry, 22(2), 104-115.

Levinson, R. (1997). PEAT: The planning and execution assistant and training system. Journal of Head Trauma Rehabilitation, 12(2), 769-775.

LoPresti, E., Mihailidis, A., & Kirsch, N. (2004). Assistive technology for cognitive rehabilitation: State of the art. Neuropsychological Rehabilitation, 14(1/2), 5-39.

Norman, R. M., Malla, A. K., Cortese, L., Cheng, S., Diaz, K., McIntosh, E., et al. (1999). Symptoms and cognition as predictors of community functioning: A prospective analysis. Am J Psychiatry, 156(3), 400-405.

Patterson, T L., Goldman, S., McKibbin, C. L., Hughs, T., & Jeste, D.

V. (2001). UCSD performance-based skills assessment. Schizophrenia Bulletin, 27(2), 235-245.

Sablier, J., Stip, E., & Franck, N. (2009). Cognitive remediation and cognitive assistive technologies in schizophrenia. L'Encephale, 35, 160-167.

Sablier, J., Stip, E., Franck, N., Giroux, S., Pigot, H., Moreau, J., etal. (2007). Study on the convivial use of an electronic agenda by individuals with schizophrenia. Sante Mentale au Quebec, 32(2), 209-224.

Stip, E., Lecardeur, L., & Ali-Sepehry, A. (2008). Computerized assessment of neurocognition in schizophrenia: An exploratory meta-analysis of CANTAB findings. European Psychiatric Review, 1(2), 48-54.

Wilson, B. A., Emslie, H. C., Quirk, K., & Evans, J. J. (2001). Reducing everyday memory and planning problems by means of a paging system: A randomised control crossover study. Journal of Neurology, Neurosurgery and Psychiatry, 70(4), 477-482.

Wilson, B. A., Evans, J. J., Emslie, H., & Malinek, V. (1997). Evaluation of NeuroPage: A new memory aid. Journal of Neurology, Neurosurgery, and Psychiatry, 63(1), 113-115.

Efficacy of PRIME, a Mobile App Intervention Designed to Improve Motivation in Young People With Schizophrenia

Danielle A Schlosser^,1,2 Ti^moth  R C'a^aellone1,3 Brand Triionn1 K”evin Etter1,2 Silvia Venxani4

aiiiee. cosser ,inoi y. a^npeone , randy ruong , evni tter ,via vergan ,

Kiya Komaiko1, and Sophia Vinogradov1,5

1UCSF Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA; 2Healthcare and Science Division, Verily Life Sciences, South San Francisco, CA; 3Mental Health Services, San Francisco Veterans Affairs Medical Center, San Francisco, CA; 4Health and Wellness, IDEO, Palo Alto, CA; 5Department of Psychiatry, University of Minnesota, Minneapolis, MN

*To whom correspondence should be addressed; 401 Parnassus Avenue, San Francisco, CA 94134, US; tel: 415-476-8721, fax: 415-4767320, e-mail: danielle.schlosser@ucsf.edu

The onset of schizophrenia occurs during a period critical for development of social relationships and functional independence. As such, interventions that target the early course of illness have the potential to stave off functional decline and restore functioning to pre-illness levels. In this entirely remote study, people with recent-onset schizophrenia spectrum disorders (SSDs) participated in a 12-week randomized controlled trial to determine the efficacy of PRIME (personalized real-time intervention for motivational enhancement), a mobile-based digital health intervention designed to improve motivation and quality of life. Participants were randomized into the PRIME (n = 22) or treatment-as-usual/waitlist (TAU/WL) condition (n = 21) and completed assessments at baseline, post-trial (12 wk), and for people in the PRIME condition, 3 months after the end of the trial. After 12-weeks, WL participants received PRIME, resulting in a total sample of 38 participants completing PRIME. In PRIME, participants worked towards self-identified goals with the support of a virtual community of age-matched peers with schizophrenia-spectrum disorders as well as motivation coaches. Compared to the WL condition, people in the PRIME condition had significantly greater improvements in self-reported depression, defeatist beliefs, self-efficacy, and a trend towards motivation/pleas-ure negative symptoms post-trial, and these improvements were maintained 3 months after the end of trial. We also found that people in the PRIME condition had significantly greater improvements in components of social motivation post-trial (anticipated pleasure and effort expenditure). Our results suggest that PRIME has the potential to be an effective mobile-based intervention for improving aspects of mood and motivation in young people with SSDs.

Key words: randomized control trial/recent-onset schizophrenia/reward learning

Introduction

Schizophrenia is a serious and disabling disorder, but with targeted early interventions, individuals may experience functional outcomes equivalent to those living without the disorder.1-4 An increasing body of evidence suggests that motivational deficits play a critical role in determining functional outcomes in schizophrenia spectrum disorders (SSDs).5,6 These deficits in adaptive goal-directed behavior encompass a range of underlying component processes, including difficulty learning from rewarding outcomes,6,7 diminished anticipation of pleasure for rewarding outcomes,8 as well as a reduction in effort expended to obtain rewarding outcomes.9,10 These impairments have been observed to be less severe early in the course of illness, suggesting it might be an ideal time to intervene in order to stave off further decline and disruptions to functioning during a critical period of development.

Utilizing technology, such as smartphone apps and web-based platforms, to deliver behavioral interventions is a promising area of research and has been found to be a feasible and effective approach to early intervention in psychosis.11-14 In addition to using technology platforms to deliver interventions, digital tools have been successfully adopted with adherence rates typically high (~80%) as well as useful for measuring precise phenotypic features of psychosis, which has significantly enhanced our understanding of individuals living with psychosis.15,16 Deploying interventions using ubiquitous technology may help make care more accessible and is a particularly important methodology given that individuals with schizophrenia typically experience motivational deficits that may disrupt engagement in traditional delivery systems of care, such as weekly psychotherapy visits.17,18 In addition to harnessing technology to deliver interventions, it is possible to utilize tech-enabled platforms to remotely conduct clinical trials—an approach that may make participation in clinical trials more accessible. Equally important to addressing engagement and access is ensuring that digital interventions are rigorously evaluated for their effectiveness and specific indications. While some digital interventions may be sufficient as a stand-alone care option (ie, self-management tools for depression), in schizophrenia, digital interventions will likely be considered adjunctive to existing care. Digital interventions may be particularly well suited to either augment existing approaches or target domains of the illness (ie, cognitive and motivational deficits), which may be difficult to treat using traditional approaches.

Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center 2018.


Page 1 of 11


The purpose of this study was to test the efficacy of a new mobile intervention called PRIME (personalized real-time intervention for motivational enhancement), which was designed to improve motivational impairments early in the course of schizophrenia. PRIME is a mobile app intervention that includes a peer community, goal and achievement tracking, and cognitive behavioral therapy (CBT) based coaching. The intervention was designed to target the motivational system by utilizing social reinforcement to engage and sustain goal-directed behavior. The targeting of motivated behavior was hypothesized to require successful engagement of the various component process of reward processing, known to be disrupted in psychosis spectrum disorders. This study was conducted after our team demonstrated the feasibility and acceptability of the intervention in a pilot sample, involving 10 participants in a trial.11 In this randomized controlled trial, the delivery of PRIME over a 12-week period was compared to a treatment-as-usual/waitlist (TAU/WL) control group. We hypothesized that participants in the PRIME condition would experience significantly greater improvements in self-reported and task-based motivational impairments, relative to the TAU/WL condition. A secondary aim of the study was to test the feasibility of conducting an entirely remote clinical trial for individuals with schizophrenia.

Methods

Recruitment

Participants were recruited remotely using Craigslist, online message boards, and flyers posted in clinics and doctor’s offices. Study investigators also listed the study on the UCSF School of Medicine Clinical Trials website and the lab website, and directly contacted other research labs. Interested participants contacted the study team and were screened and enrolled entirely remotely from 13 states across the United States (California: n = 23, Texas: n = 3, Tennessee: n = 2, New York: n = 2, Nevada: n = 1, Idaho: n = 1, Arkansas: n = 1, Maryland: n = 1, Virginia: n = 1, Washington: n = 1, North Carolina: n = 1, South Carolina: n = 1, Colorado: n = 1), as well as 2 countries outside of the United States (Canada: n = 3, Australia: n = 1). Inclusion and exclusion criteria for participants were: (1) meeting DSM-IV-TR criteria for a diagnosis of schizophrenia, schizophreniform, or schizoaffective disorder, (2) being in the early course of illness (defined as being within the first 5 y of formal diagnosis), (3) being between the ages of 16 and 36, (4) not meeting DSM-IV-TR criteria for substance dependence within the 6 months prior to starting the study, (5) being clinically stable (no changes in outpatient status or medication) for at least 1 month prior to starting the study, (6) being able to provide informed consent, (7) not having history of neurological disorders or serious head trauma, (8) being fluent in English, and (9) having an estimated IQ > 70 as measured by the Wechsler Test of Adult Reading (WTAR).19 Demographic information, such as age, diagnoses, and years of education, as well as utilization of treatment resources (eg, therapy, psychiatric services), can be found in table 1.

Remote Data Collection Procedures

For all participants, informed consent documents were built and sent via Qualtrics Insight Platform (Provo, UT). Participants were guided through the informed consent process over the phone with a trained research assistant and provided their informed consent by checking a box and typing their name in the online document. Self-report measures and behavioral tasks were also administered via Qualtrics Insight Platform. All interview-based clinical assessments were conducted via FaceTime or Skype.

Study Design

This 12-week randomized control trial (RCT) tested the efficacy of the second iteration of PRIME, including modifications informed by the results of our pilot study. Participants were randomized to either receive PRIME or a TAU/WL control condition. Participants were compensated for their time to complete study-related assessments ($20/h) but were not paid for their participation in the intervention. Study evaluators were blind to treatment condition.

Study Sample

Of the participants who completed an initial phone screen of eligibility (n = 140), 77 were excluded. The remaining 63 potentially eligible participants then completed a thorough intake assessment to confirm eligibility, with a total of 43 participants being randomized to PRIME (n = 22) or a WL condition (n = 21). Participants in the WL condition were offered the opportunity to receive PRIME after 12 weeks (see figure 1 for CONSORT diagram); 20 out of 21 waitlisted participants opted to receive PRIME after these 12 weeks. The assessments completed

Table 1. Demographic and Clinical Characteristics

PRIME (n = 22) Mean (SD)

Waitlist (n = 21) Mean (SD)

t or x2 (P)

Age (y)

24.32 (2.6)

23.79 (4.5)

.68

% Male

60%

65%

.70

Education (y)

14.08 (2.3)

13.37 (1.8)

.24

SSD diagnoses

.93

Schizophrenia

12 (55%)

11 (52%)

Schizoaffective

8 (36%)

8 (38%)

Schizophreniform

2 (9%)

2 (10%)

Duration of illness (y)

2.32 (1.4)

2.73 (1.6)

.32

Racial background (%)

Caucasian

50%

55%

.86

Asian

15%

20%

African American

12.5%

5%

Other

22.5%

20%

% Seeing therapist

70%

70%

.95

% Seeing psychiatrist

57.5%

65%

.62

WTAR FSIQ

111.89 (8.9)

113.77 (10.0)

.48

PANSS

Positive symptoms

7.17 (3.8)

8.10 (4.6)

.40

Negative symptoms

12.54 (5.4)

12.45 (5.1)

.95

CPZ equivalents

273.03 (295.4)

298.50 (389.1)

.81

Note: PRIME group is combined sample of both conditions. PRIME, personalized real-time intervention for motivational enhancement; WTAR FSIQ, wechsler test of adult reading full scale IQ; PANSS, Positive and Negative Syndrome Scale; CPZ, Chlorpromazine; SSD, schizophrenia spectrum disorder.


at this time doubled as final WL condition scores and as baseline scores prior to entry into PRIME. All participants met DSM-IV-TR20 criteria for a SSD, evaluated using the Structured Clinical Interview for DSM-IV-TR Disorders.21 Participants were given the option of borrowing a smartphone from the researchers, and 12 of the 43 participants exercised this option. Of the 43 participants randomized to the PRIME or WL conditions, 37 (86%) were taking an antipsychotic medication at the time of the study. In order to compare the antipsychotic burden across people with an SSD, we calculated chlorpromazine (CPZ) equivalents using a standardized conversion table.22 Clinical and demographic characteristics for both conditions may be found in table 1.

PRIME Description

PRIME was designed in collaboration with IDEO, a global design, and innovation firm, to implement a human-centered design process and make the platform appealing to users and address their needs. With PRIME, participants joined a supportive online environment where they selected and documented progress on small, self-determined goals in the domains of health/wellness, social relationships, creativity, and productivity. When participants set up a PRIME account, they selected longterm goals from a 36-item list, which included goals such as “deepen my relationship with my family” and “feel more relaxed.” A feature was developed later in the study permitting users to add to and modify the goals they selected during account set up. When using the goal feature later, participants chose one of the long-term goals they indicated interest in during setup. This triggered a display of brief challenges (able to be completed in a day) that contributed to that goal, such as “offer to help a family member with a chore, like shopping or cleaning!” or “invite a family member to do something fun with you.” Each long-term goal contained more than 15 suggested challenges on average. Participants sequentially viewed these suggested challenges and had the ability to create a custom challenge, which they could manually enter. Participants viewed the suggested challenges one at a time and, as a participant completed several challenges for the same goal, the sequence adapted to display more ambitious options. Suggestions laddered from challenges like “listen to relaxing music for five minutes...” to “go to a yoga class.” but participants were always able to view and select the easier-to-accomplish challenges at any time. Participants were given automatic reminders of the challenges and indicated when they completed the challenges they selected. Completion prompted participants with an opportunity to post a quick “accomplishment moment” which they shared with their coach and the PRIME community.

PRIME provides users with motivation coaches; masters-level clinicians who use evidence-based interventions drawn from CBT, behavioral activation, mindfulness, and psychoeducation to help participants overcome the daily obstacles that hinder goal progress and improve health outcomes. Additionally, the PRIME community provides a platform for users to interact with one another. Users may send messages directly to each other and can also


capture and share positive, spontaneous moments in their daily life with the whole PRIME community.

When first-time participants signed in to the app, a research assistant guided them through the process of creating a user profile. Participants created a username, uploaded a profile picture, selected their interests, goals, and symptoms, and wrote a short bio. Once a user was registered to the app, an assigned motivation coach sent the new participant a welcome message and an offer to support the participant in achieving his/ her goals. The coach informed his/her assigned participants that he/she would be available to message with them “most days” per week, but would modify the frequency depending on their preference, clinical issues, and overall progress towards goal achievement. When possible, messaging between coaches and participants was synchronous (ie, in real-time) to facilitate intervention development and implementation. Participants could also request to speak with coaches on the phone or via FaceTime. Goal achievement was measured by the number of challenges completed in each goal domain. Participants in the PRIME condition were encouraged to use the app daily, whether it be to message with coaches and/or peers or complete challenges. However, the minimum expectation for participation in this intervention was logging into PRIME at least 1x/ wk over the 12-week period.

PRIME Outcome Measure

The primary outcomes for this trial were changes in components of motivated behavior from baseline to 12-weeks using a modified version of the Trust Task.23-25 In line with recent models of motivation impairment in people with schizophrenia8,26 the Trust Task was designed to assess 3 components of motivation: reward learning, anticipated pleasure, and effort expenditure. The initiation and execution of motivated behavior involves each of these components, and with PRIME participants receive support in their pursuit of goal-directed behavior as well as feedback about their performance. As such, the Trust Task, which assesses these specific motivation components in a social context, can provide an objective index of how engagement with PRIME generalizes to improvements in motivation for social interaction more broadly.

During this task, participants interacted with 4 simulated social partners identified by name and a dynamic video of them expressing a facial display. Participants indicated their anticipated pleasure from the outcome of the interaction (1 to 7 scale). To measure reward learning, participants decided how many points to send to a social partner (between 0 and 10) using the keyboard. The amount of points sent by the participant was then quadrupled, and social partners returned a percentage of the quadrupled amount (0% to 100%), with both the participant and social partner’s percentage shown on the screen. Thus, the key variables related to trust for each trial were the amount of points sent by a participant (represents how much he/she trusted a social partner) and the percentage of points returned by the social partner (represents the extent to which participant trust was reciprocated). Intact reward learning would mean giving more points to trustworthy and fewer points to untrustworthy social partners. Finally, participants could influence the likelihood of interacting with this social partner again in the future by expending effort in the form of repeated key presses. Participants could repeatedly press a specific key to increase the likelihood, a different key to decrease the likelihood, or simply choose to do nothing for the duration of the 6-second response window if they did not have a preference. We averaged the number of key presses across the response window to create an index of the number of key presses per second per participant. Social partner behavior was predetermined so that interactions with 2 partners resulted in positive outcomes (average return double the amount sent) while interactions with the other 2 social partners resulted in negative outcomes (average return half the amount sent). Participants interacted with each social partner 8 times for a total of 32 trials. In previous studies, both the average amount of trust placed24 and the average amount anticipated pleasure23 during interactions with trustworthy social partners has been shown to be positively associated with social functioning in people with schizophrenia.

PRIME Outcome Assessment: Secondary

In addition to our primary outcome, we assessed self-reported defeatist beliefs and change in motivation using the Motivation and Pleasure-Self Report scale (MAP-SR),27 which is a self-report version of the Motivation and Pleasure scale from the Clinical Assessment Interview for Negative Symptoms.28 Second, we assessed real-world functioning in independent living, work, family, and social domains using the interview-based Role Functioning Scale (RFS).29 In addition, we assessed quality of life in social and vocational domains using the interview-based Quality of Life Scale - Abbreviated (QOL-A).30 We also assessed defeatist beliefs about successfully performing goal-directed behavior using the 15-item subscale31 of the Dysfunctional Attitudes Scale,32 depression symptom severity with the Beck Depression Inventory, Second Edition (BDI),33 and self-efficacy with the Revised Self-Efficacy Scale (R-SES).34 All self-report measures had acceptable internal consistency (a > .80). We also assessed positive and negative symptoms using the Positive and Negative Syndrome Scale (PANSS).35

The same remote assessment schedule was used for participants in both conditions and included clinical evaluations at baseline and 12-weeks. Because participants in the WL condition were given the option to join the PRIME condition immediately after the 12-week time point, we did not conduct a 3-month post-study assessment with these participants. Since our primary outcome was changes in motivated behavior between study conditions, and we did not conduct a 3-month assessment for WL participants, the Trust Task was not administered at this time point. Outcome evaluators in the RCT were blind to condition.

PRIME Acceptability

We assessed PRIME acceptability during an exit interview at the 12-week time point (post-trial) where participants rated their satisfaction with the specific features of PRIME, such as the ability to interact with peers and the different goal categories, on a 1 (not at all) to 10 (very much) scale. We also assessed retention in the trial as a measure of acceptability.

PRIME Feasibility

To evaluate feasibility, we assessed the following use metrics: login frequency (average number of days logged in per week), average number of challenges completed (both overall and by individual challenge category), challenge completion percentage, and the average number of peer and coach interactions. Interactions included direct messaging on PRIME as well as commenting on and liking content posted to the community moments feed. To further understand how participants were engaging with the PRIME platform, we evaluated “active use rate.”11 To do this, we added together the average number of challenges completed, peer, and coach interactions and divided this total by the number of weeks the participant had access to PRIME. Thus, a value of 2.3 would mean that a participant was active on PRIME 2.3 times/wk. Passive use was defined as logging into the app, but not posting a moment, completing a challenge or interacting with peers or coaches. Thus, a participant may log in to the app 4 days per week but actively engage with the features of the app 2 times/wk.

Data Analysis Plan

We used an intent-to-treat analysis, and thus all participants who completed baseline assessments were randomized and included in the analyses. All analyses involving the PRIME condition included both participants initially randomized into this condition as well as WL condition participants who received PRIME after 12-weeks. First, we examined whether any demographic variables were related to baseline motivation (MAP-SR, Trust Task components) or functioning (RFS, QOL-A) and whether any demographic variables, baseline motivation, and functioning were related to PRIME utilization using correlations. To determine PRIME acceptability, we examined the average ratings from the PRIME satisfaction survey administered at the 12-week time point for overall

satisfaction as well as the most and least popular PRIME features. Furthermore, we also reviewed qualitative feedback from the PRIME exit interview. To investigate PRIME feasibility, we examined descriptive statistics for the following PRIME metrics: login frequency, challenges completed, spontaneous and goal achievement moments, peer and coach interactions, and active use rate.

To investigate the effect that PRIME had on our primary and secondary outcomes, we conducted ANCOVAs comparing changes from baseline to the 12-week time point for participants in the PRIME and WL conditions. ANCOVA allowed us to examine group differences in our outcomes of interest at the 12-week time point while controlling for baseline scores. For participants in the WL condition, the 12-week time point data also served as their baseline time point data for when they entered the PRIME condition.

To test whether changes in our primary and secondary outcomes persisted at the 3-month time point, we conducted paired samples Etests between 12-week minus baseline and 3-month minus baseline change scores, with the exception of the Trust Task, which was only administered at the baseline and 12-week time points. To explore the degree to which specific aspects of PRIME use were associated with improvements in our primary and secondary outcomes, we computed correlations between change scores in our outcome measures (12-wk time point minus baseline) and PRIME login percentage (passive use), PRIME active use rate, total number of coach interactions, total number of peer interactions, and total goals completed.

Results

Five participants in both the PRIME and WL conditions dropped out while using PRIME or were unreachable for post-intervention assessments, and an additional 6 were unreachable or otherwise did not complete followup assessments 3 months later (figure 1). Since participants in the WL condition were able to use PRIME after 12-weeks, analyses comparing changes at the 12-week time point have n = 38 in the PRIME condition (19 originally randomized into PRIME plus an additional 19 WL participants who received PRIME after 12-wk) and n = 21 in the WL condition. The sample size for analyses looking at the 3-month time point in the PRIME condition is n = 32, which means that 74% (32/43) were retained for the duration of the trial and follow-up period.

Participant demographics were not related to baseline clinical symptoms or PRIME use metrics. Further, there were no differences in demographic or baseline symptoms between participants randomized into either condition. Both PRIME use metrics (login percentage, interactions) and the effort expenditure component of the modified Trust Task were not normally distributed, so we conducted a root transformation on these data.

The Effect of PRIME on Our Primary Outcome: Motivated Behavior

To investigate the effects of PRIME on our primary outcome, we compared the PRIME and WL conditions on changes in the Trust Task from baseline to post-trial (12-wk time point). We found a significant difference between conditions in anticipated pleasure during the modified Trust Task, F(1,56) = 4.75, P = .03, with participants in the PRIME condition showing a greater increase from baseline to 12 weeks compared to WL, t(55) = -2.39, P = .02, d = 0.64 (figure 2a). Similarly, we found a significant difference between conditions in effort expended to increase the likelihood of future social interactions with positive outcomes, F(1,56) = 4.66, P = .04, with participants in the PRIME condition showing a greater increase from baseline to 12 weeks compared to WL, t(55) = -2.17, P = .03, d = 0.58 (figure 2b). Furthermore, we found a trend towards significant improvement in learning from positive outcomes during the modified Trust Task, F(1,56) = 3.53, P = .07. There were no significant differences in effort expended to decrease the likelihood of future interactions with positive outcomes, nor in changes in components of motivation for interactions with negative outcomes (Ps > .20).

The Effect of PRIME on Secondary Outcomes: Self-reported and Clinically Assessed Motivation, Symptoms, and Functioning

Next, we investigated the effects of PRIME on our secondary outcomes: self-reported defeatist beliefs, motivation, depression, self-efficacy, and clinically assessed positive and negative symptoms and functioning. We found significant differences between conditions for defeatist beliefs, F(1,57) = 5.58, P = .02, with participants in the PRIME condition showing a greater decrease from baseline to 12 weeks compared to WL, t(56) = 2.22, P = .03, d = 0.59 (figure 3a). We found similar effects for depression symptoms, F(1,56) = 7.06, P = .01, and self-efficacy, F(1,55) = 5.76, P = .02, with the PRIME participants showing greater improvements from baseline to 12 weeks, t(53) = -2.30, P = .03, d = 0.63 (figure 3b), and t(56) = -2.39, P = .02, d = 0.64 (figure 3c), respectively. Importantly, comparing changes from baseline to 12-weeks and baseline to 3-months, we found no differences, suggesting that these gains were maintained 3 months post-trial. We also found a trend towards significant improvement on the MAP-SR, F(1,57) = 3.79, P = .06. There were no group differences in changes in positive or negative symptoms (PANSS), quality of life (QOL-A), or functioning (RFS) from baseline to the 12-week time point, nor the 12-week to the 3-month time point (Ps > .28).

As an exploratory follow-up, we examined whether the same pattern of findings was true for only participants in the WL condition who then received PRIME. Generally


consistent with our primary outcomes findings for the combined sample (PRIME first and WL first), we found participants who were randomized to the WL condition had significantly greater learning from positive outcomes following 12 weeks of PRIME, F(1,37) = 4.53, P = .04. We also found trends toward greater anticipated pleasure, F(1,37) = 3.64, P = .07, and effort expended to increase the likelihood of future social interactions with positive outcomes following 12 weeks of PRIME, F(1,37) = 3.28, P = .08. Our secondary outcome findings were less consistent with the combined sample as participants who were randomized to the WL condition had significantly greater improvements in depression, F(1,38) = 3.98, P = .05, and defeatist beliefs, F(1,38) = 4.39, P = .04, but not the MAP-SR (P = .41) or self-efficacy (P = .10).

Exploring Whether PRIME Use is Related to Changes in Primary and Secondary Outcomes

To explore how PRIME improved symptoms and behavior, we explored correlations between PRIME metrics (PRIME login percentage, PRIME active use rate, total number of coach interactions, total number of peer interactions, and total goals completed) and changes in our primary and secondary outcomes from baseline to posttrial (12-wk). We did not find any significant correlations between PRIME metrics and change in our outcomes measures (Ps > .10).

PRIME Acceptability

Mean overall satisfaction with PRIME for the entire sample, as rated during the exit interview administered at the 12-week post-assessment, was 8.21 (SD: 1.9). Some of the comments made by participants when asked about how PRIME impacted their lives included: “(PRIME) helped me see that ‘you’re not the only one’ by seeing others do well and be able to get better”, and “It was good to have coaches to speak to when I needed it. Working with my coach really helped me work on testing some paranoid beliefs and teaching me how to test those on my own as well. ...Helped reduce suicidality primarily through instilling some level of hope, which came from feeling connected to a larger group and resource. This connection felt like a solid foundation. Decreased helplessness too.”. The most popular PRIME feature was the ability to directly message coaches (M: 8.38, SD: 2.5), and the least popular PRIME feature was the ability to track your mood (mean: 6.33, SD: 2.4).

PRIME Feasibility

Average PRIME use data per participant (login frequency, challenge completion, interactions) can be found in table 2 and are presented separately for participants randomized to receive PRIME first or after 12-weeks in the WL condition. On average, participants logged in a little over 4 d/wk. Over a 12-week period, participants were highly engaged in the platform, with 5152 direct messages sent from participants to coaches. In terms of peer-to-peer interactions, participants initiated interactions with each other a total of 497 times. Participants initiated about 10 interactions with coaches for every initiated peer interaction. All 38 participants initiated at least 1 message to a coach and 13 (33%) initiated more than the average of 128.8 coach interactions. However, there was a considerable amount of variability in coach messaging, with the range for initiated coach interactions being 12 to 574. Participants completed an average of


1.5 challenges per week. Health/wellness challenges were the most popular at about 1 challenge completed every 2 weeks, followed by creativity, social challenges, and productivity challenges of which participants completed approximately 1 every 3 to 4 weeks. Challenge completion percentage was high (88%), suggesting that participants had little difficulty completing the challenges that they set.

PRIME activity, defined as both the number of challenges completed and number of messages sent, was highest during the first month. However, engagement with the PRIME features dropped after the first month of the trial before leveling out over the second and third months. To illustrate this point, participants initiated an average of 10.3 challenges in the first month compared to 4.1 and 3.0 in the second and third months. A similar pattern was observed for messaging with coaches, with participants initiating an average of 53.9 messages in the first month compared to 34.2 and 31.8 messages in the second and third months. Three participants discontinued PRIME use right before or during the third month of the trial, which could have contributed to this pattern of decreased engagement over time. Taken together, the relative maintenance of PRIME use over the course of the second and third months may reflect a more stable, long-term engagement with the application.

Discussion

This study demonstrated that PRIME is a feasible, acceptable, and efficacious intervention for improving mood and motivation in young people with an SSD. The overall 74% retention rate for the treatment (and 88% retention post-intervention), demonstrated that this intervention

Table 2. PRIME Utilization Data

PRIME Use Metric (Range)

PRIME First (n = 19)

Waitlist First (n = 19)

Average logins per week (1.2-7 d/wk)

4.03 (1.4)

4.10 (1.5)

Challenge completion rate (50%-100%)

91.47 (12.2)

83.58 (21.0)

Average number of user-initiated peer interactions

Comments (0-29)

4.54 (7.0)

4.58 (5.5)

Likes (0-74)

8.91 (20.2)

11.05 (18.4)

Messages (0-131)

9.91 (16.5)

14.95 (30.4)

Total (0-174)

23.36 (39.2)

30.58 (37.0)

Average number of user-initiated coach interactions

Comments (0-39)

6.50 (11.6)

8.63 (10.5)

Likes (0-50)

6.95 (14.6)

5.84 (9.7)

Messages (12-574)

150.68 (142.0)

100.74 (85.0)

Total (17-581)

164.14 (141.0)

115.21 (95.0)

Challenges completed

Overall (1 to 52)

14.91 (13.1)

18.11 (15.4)

Health/wellness (0 to 27)

4.68 (6.3)

6.74 (6.2)

Social (0 to 13)

3.00 (3.2)

3.79 (4.3)

Creativity (0 to 20)

3.77 (4.15)

4.79 (6.1)

Productivity (0 to 12)

3.45 (3.5)

2.79 (3.1)

Active use rate (0.27 to 4.97)

1.76 (1.3)

1.94 (1.4)

Note: PRIME, personalized real-time intervention for motivational enhancement.


was very well tolerated. Participants rated their overall satisfaction with PRIME highly, which was demonstrated by the degree of engagement observed in the app. Participants, on average, used PRIME a little over 4 d/ wk and sent over 5000 messages to coaches and approximately 500 to their peers. Many participants noted that it was the first time they had seen or interacted with other young people with an SSD, and they particularly appreciated being able to have on-demand coaching, as demonstrated by the qualitative feedback and this feature being rated as the most satisfying.

This preliminary examination of efficacy found that participants in PRIME experienced significant improvements in depression, defeatist beliefs, self-efficacy and several important components of motivation, such as reward learning, anticipated pleasure, and effort expenditure. As such, PRIME appeared to function as a behavioral activation intervention. The improvements in the domains of motivation were a specific focus in the initial design of PRIME,11 which emphasized engagement in goal-directed behavior, capturing images and cataloging positive experiences, and sustaining engagement (encouraged by social reinforcement) in sharing experiences in the PRIME community. Furthermore, we showed that PRIME specifically targets and engages components of motivation that work together to produce motivated behavior. Indeed, our findings show that PRIME increased learning from positive outcomes, anticipated pleasure for positive outcomes, and expenditure of effort to obtain future positive outcomes, which are components of motivation not typically improved by medications nor in-person psy-chotherapy.36,37 As such, PRIME may act as an important adjunctive intervention to treatment approaches that are usually more focused on treating the positive psychotic symptoms, and offer a more holistic approach to improving outcomes for people with an SSD.

There were several limitations to this study, including a relatively small sample size that may not have been representative of the population with SSD or adequately powered to determine whether deploying PRIME would be successful in improving other important clinical outcomes, such as role or social functioning. Secondly, the use of a TAU/WL control condition for this study did not allow for an understanding of the relative effect of PRIME compared to other types of mobile interventions or treatment approaches. And lastly, the relatively short follow up period (3 mo) may not have been long enough to conclude whether the effects of PRIME would translate into longer term and clinically meaningful outcomes. Further research is needed to improve our knowledge about the moderators of outcomes as well as refine our understanding of who may benefit more or less from this intervention approach.

To our knowledge, this is the first study to demonstrate that a mobile intervention may improve critical domains of impairment in SSD. Further, this study was conducted entirely remotely and successfully recruited, enrolled, and engaged young people all over the United States, Canada, and Australia. By using this methodology, clinical trials may be conducted more efficiently and potentially recruit an even more diverse sample than in academic settings. Lastly, by implementing a humancentered design process, we ensured that PRIME was able to pair a scientific foundation with an approach that resonated with the needs of young people with an SSD.

Funding

This work was supported by the National Center for Research Resources at the National Institutes of Health (R34 MH100399).

Acknowledgment

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

S. Creating live interactions to mitigate barriers (CLIMB): a mobile intervention to improve social functioning in people with chronic psychotic disorders. JMIR Ment Health. 2016;3:e52.

BMC Research Notes

RESEARCH NOTE                                      Open Access

Improving adherence in mental health service users with severe mental illness in South Africa: a pilot randomized controlled trial of a treatment partner and text message intervention vs. treatment as usual

f~\ f~\ /-'I rvx —s Kx Ci k'x f~\ 1/ z*~\ 1^^^ I I lz I o kv\ kv\ i z*“'i Px 1 C i i m ^ \ /*“\ KA'“xI|1d^^^k\A/iIIi^ kv\ c A r h m                           I kh i/“ kz'~\T|“2

GOOdman siocko , nenK lemmingn , sumaya Mall , Peter vviiiiams-Asnman , Graham i mornicrort , Ezra S. Susser3, CricK Lund1,2, Dan J. Stein34 35 and Peter D. Milligan35

Abstract

Objectives: Medication non-adherence is a significant problem in treatment of severe mental disorders and is associated with poor clinical outcomes and increased demand on services. TasK-shifting interventions incorporating mobile health may improve adherence in mental health service users in low- and middle-income countries. Seventyseven participants were recruited from a psychiatric hospital in Cape Town, with 42 randomized to receive the intervention and 35 to treatment as usual. Intervention pairs underwent treatment-partner contracting and psychoeducation, and received monthly text message reminders of clinic appointments. Primary outcomes were intervention acceptability and feasibility. Secondary outcome for efficacy were adherence to clinic visit; relapse; quality of life; symptomatic relief and medication adherence.

Results: Treatment partner and psychoeducation components were acceptable and feasible. The text message component was acceptable but not feasible in its current form. Efficacy outcomes favoured the intervention but did not reach statistical significance. A treatment-partner intervention is acceptable and feasible in a low- and middle-income setting. Work is needed to ensure that additional components of such interventions are tailored to the local context. Appropriately powered efficacy studies are needed.

Trial Registration PACTR PACTR201610001830190, Registered 21 October 2016 (Retrospectively registered)

Keywords: Mental health, TasK-shifting, Treatment partner, Adherence, Text message, Mobile health

Introduction

Poor medication adherence is a major problem in the treatment of severe mental disorders and is associated with poor clinical outcomes and increased demand on services [1-3]. Thus, there is significant interest in developing interventions to improve medication adherence in this group [4, 5].

Mental health services are under-resourced globally, with low- and middle-income countries (LMIC) facing a particular challenge [6]. One approach that seeks to address the shortfall is task-shifting, which refers to the delivery of evidence-based interventions by non-special-ist workers [7-9]. Treatment partner interventions may represent a useful task-shifting approach in adherencepromotion for mental health service users (MHSU). A MHSU is an individual accessing care, treatment and rehabilitation services via a health establishment for the purpose of enhancing his or her mental health status [10].


BioMed Central


© The Author(s) 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.Org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/ publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.


Telephone prompts have shown promise in encouraging treatment adherence and clinic attendance [4, 11]. Mobile health (m-health), defined as medical and public health practice supported by mobile devices, is gaining popularity worldwide [12, 13]. While m-health approaches require further investigation to establish efficacy, they have been adopted in maternal and child health, and chronic diseases [12, 14, 15]. This technology has not been widely used in adherence-focused mental health interventions in LMIC.

Approaches to adherence-support have emphasized the importance of health literacy, problem-solving, and social support, as well as the potential of telephone prompts [45]. It is however unclear whether an approach incorporating these components is feasible in a LMIC setting.

Our two-arm non-blinded prospective pilot randomized controlled trial evaluated the acceptability and feasibility of a treatment partner and text message intervention in supporting adherence in people with severe mental illness.

Main text

Methods

We recruited over 2 years from Valkenberg hospital (VBH), which offers public psychiatric services in Cape Town, and at 15 psychiatry clinics in the hospital’s catchment area. VBH has 116 male, and 84 female beds with majority of patients having a diagnosis of severe mental illness, constituting a significant service and disability burden [16]. The psychiatry clinics providing post-discharge care in the VBH catchment area are run by mental health nurses.

We screened clinical folders of MHSU in the pre-discharge wards at VBH. We included MHSU diagnosed with schizophrenia spectrum disorder, substance induced psychotic disorder, and bipolar mood disorder type I. We approached eligible MHSU and obtained informed consent from those willing to participate after being informed about the study. Exclusion criteria included: (a) a diagnosis of psychotic disorder due to a general medical condition, dementia, moderate to severe intellectual disability; or (b) suicidality; or (c) an inability to give informed consent.

Participants were randomized to either intervention or treatment as usual (TAU), using a randomization sequence generated by an external statistician. We ensured allocation concealment by using opaque, sealed sequentially numbered envelopes.

Intervention participants nominated a treatment partner from within their social support network. An appointment was set at a date preceding the participant’s discharge from VBH for intervention procedures to be initiated.

Routine care and study activities are described in Table 1. The intervention incorporated TAU with the addition of (1) a treatment partner contracting and psychoeducation session and (2) text message reminders of clinic appointments. Intervention development was informed by focus group and in-depth interview work (published elsewhere), where we found there was a need for caregiving relationships to be negotiated to protect he MHSU’s autonomy [17].

The treatment partner contracting and psychoeducation session was conducted with the intervention pair at VBH on the agreed-upon date. Psychoeducation focused on the participant’s diagnosis and was based on the national institute for mental health and the VBH psychoeducation guidelines, with input from specialist psychiatrists at VBH [18]. Psychoeducation addressed mental health literacy needs uncovered during our formative work and included psychiatry clinic follow-up processes. An individualized participant/treatment partner relationship was negotiated and agreed upon, detailing a communication and adherence-support approach.

Upon discharge, participant information was loaded onto a text message system using one enrolment handset. Fifteen fieldworker handsets were programmed for use by the nurses. Once enrolled, the fieldworker handsets were remotely populated with the relevant participant information. Once discharged, the intervention pair received a text message, indicating the appointment clinic and date. Another text message was sent 1 week before the appointment. TAU participants received no text messages. When participants arrived at the clinic the nurse used the handsets to mark the participant’s attendance, and to schedule a follow-up appointment on the system, which then sent similar notifications to those sent upon discharge. Nurses scheduled 2 follow-up appointments in total.

Assessment measures

Participants underwent a structured clinical interview for diagnosis of axis-I disorders (SCID-I) to confirm diagnosis, which included a global assessment of function scale (GAF) and clinical global impressions scale (CGI). We administered the positive and negative syndrome scale (PANSS) to measure severity of psychotic symptoms. These scales have been widely used locally in genetics and treatment studies [3, 19-22]. The Camberwell assessment of needs scale (CAN) was used to measure met and unmet needs [23]. The EUROQOL was used to measure functional level, and the medication adherence rating scale (MARS) was used as a measure of medication adherence [24, 25].

We used the text message system to track intervention group attendance at first clinic visit. The attendance

Table 1 Routine care, timelines and corresponding study activities


Stage in MHSU care

MHSU care activity

Routine care

In pre-discharge ward

The MHSU receives standard pre-discharge care.

This includes

Clinical review

Finalizing of treatment plan

Psychosocial rehabilitation programmes Individual and group psychoeducation Discharge planning by the clinical team, encompassing identification of which CHCthe MHSU will be discharged to and when, as well as arrangement of post-discharge psychosocial support for the MHSU

On day of discharge

The MHSU discharged from inpatient care with referral letter detailing course of admission, diag nosis, treatment plan, review CHC and date

1 week before first clinic appointment

No activity

At first clinic appointment

The MHSU is reviewed by the mental health nurse or psychiatry registrar at the psychiatry clinic of CHC

Medication is renewed or modified

A follow up appointment is issued

The MHSU's CHC clinical record clinical record updated to indicate

Attendance for review

Clinical status

Medication review and prescription

Next scheduled appointment date

The MHSU collects his or her medication and leaves the CHC

After the first clinic appointment

The MHSU is reviewed on the scheduled review date and the process described above at first clinic appointment is repeated for subseguent visits

3 months after discharge

Routine clinical care is continued


Corresponding research activity

Treatment as usual = routine care PLUS

Intervention = treatment as usual PLUS

  • 1. We obtained informed consent from eligible patients

  • 2. We conducted recruitment activities including diagnostic and baseline measures as described in the text

  • 3. We randomized recruited participants to receive either treatment as usual or the intervention

  • 4. We informed participants which group they had been randomized to

  • 5. Participants who had been randomized to receive the intervention selected a treatment partner as described in the text

  • 6. These selected treatment partners were contacted telephonically, consented and a date was set for the psychoeducation and contracting session

  • 7. The psychoeducation and contracting session was conducted at Valkenberg Hospital for the participant and treatment partner pair

8. Participants were enrolled onto the text message platform

  • 9. The first text message was sent to the participant/ treatment partner pair containing details of first clinic appointment as per discharge treatment plan

  • 10. Fieldworker handsets received updated participant review schedules

No activity

11. A text message reminder was sent to the par-ticipant/treatment partner pair containing same details as first message

No additional activity

  • 12. On arrival, mental health nurse checks the participant as present

  • 13. The mental health nurse asksthe participant how many days of medication have been missed since discharge from hospital and enters this information on the fieldworker handset

  • 14. The mental health nurse enters the next scheduled clinic review appointment date onto the text message platform via the fieldworker handset

  • 15. The participant/treatment partner pair immediately received a text message notification indicating the next appointment date with the name of the CHC as in the initial text message

No additional activity

  • 16. The participant/treatment partner pair receives

a text message reminder of the next appointment date one week before that appointment

  • 17. The process described at steps 12-16 above is repeated for three visits in total


9 months after discharge



of both groups was additionally captured retrospectively from the attendance registers at the clinics. All participants needed to return for a follow-up study visit

At 9 months, readmissions were noted via Clinicom, a secure online platform used to track health service users accessing public health services in Cape Town, which maintains detailed notes regarding assessments, admissions, and treatment (see Additional file 1).

We expected that the intervention would be acceptable and feasible, and result in increased first clinic visits and reduced 9-month readmissions. We expected an improvement in medication adherence and quality of life, a reduction of needs and symptomatic improvement.

Statistical analysis

Primary outcomes were intervention acceptability and feasibility. Efficacy outcomes, which were secondary, were adherence to first clinic visit; relapse, defined as readmission to hospital; medication adherence; quality of life and symptomatic relief. Data for clinic visits and readmission were complete. The efficacy outcomes had incomplete outcome data as only 34 participants attended the study visit at 3 months. We report all effect measures as risk ratios or mean differences with their corresponding 95% confidence intervals. A two-tailed significance level of 5% was used throughout and analyses were conducted using Stata version 13.

Results

The participant flow diagram is available as an additional figure (see Additional file 2). Seventy-seven participants were randomized, 42 to the intervention and 35 to TAU. Efficacy outcomes were analyzed at the end of the 2-year recruitment period (n = 77). With the significant participant attrition at 3-month study follow up, we calculated that a sample size of 520 would be required to demonstrate a moderate effect.

Sample demographic characteristics are presented in Table 2.

Treatment partners included family members, partners and friends. At 3-month review, intervention participants understood their diagnosis better than TAU participants. Understanding of medication regimen and reported treatment adherence were similarly low in both groups. The psychoeducation session was seen to be superior to standard pre-discharge psychoeducation. Treatment partners showed a better knowledge of diagnosis. Intervention pairs felt that a psychoeducation follow-up would be valuable in the long-term to reinforce knowledge.

Some participants (41.2%) did not receive the text messages, either due to changed cellphone numbers or due to misplacement or theft of cellphones. Those who received the text messages all found them helpful. The nurses found the refreshing of handsets burdensome and challenging in spite of reinforcement training. Three of the fieldworker handsets were lost through theft.

All participants found the clinic easy to access and navigate. More treatment partners than caregivers found the clinic helpful in supporting adherence. The experiences of being a treatment partner and a caregiver were similar, with half in each group reporting a positive experience and the other half experiencing it as challenging [26].

The intervention was acceptable. The psychoeducation and the treatment partner components were feasible while there were significant challenges with the text message system.

Efficacy outcome data is represented in Table 3. TAU participants were more likely to miss their first clinic visit and to relapse in the 9 months following discharge. At 3 month review, TAU participants were more likely to show worsening PANNS scores, while GAF scores showed better improvement amongst intervention participants. CGI scores suggested a symptom improvement for the intervention group while the GAF scores of both groups suggested general improvement in functioning. There was a trend towards an increase in unmet needs in the TAU group vs a decline in the intervention group, while met needs were seen to increase in both groups.

Discussion

We found that (1) The treatment-partner and psychoeducation components were acceptable and feasible; (2) The text message component was acceptable but not feasible in its current form; and (3) Efficacy outcomes favoured the intervention but did not reach statistical significance.

The acceptability and feasibility of this intervention is consistent with prior literature [8]. Our approach was in line with previous caregiver focused interventions, which have targeted mental health literacy on the experience of caregiving. Family psychoeducation interventions have reduced relapse in people living with severe mental illness, while helping to meet caregiver needs [2728]. While it has been more difficult to change coping styles and reduce caregiver burden, short term interventions have improved caregivers’ knowledge and attitudes towards MHSU and mental illness [29, 30].

Table 2 Baseline variables

Total samplea (N = 77)

Intervention (N = 42)

TAU (N = 35)

Statistic (df)

p value

Mean

(SD)

Mean

(SD)

Mean

(SD)

Participant characteristics

Age

35.5

(10.2)

35.3

10.9

35.8

9.5

t =

- 0.35 (75)

0.726

N

(%)

N

(%)

N

(%)

Sex

x2 =

= 1.03 (1)

0.311

Male

55

71.4

28

66.6

27

77.1

Female

22

28.6

14

33.3

8

22.9

Ethnicity

x2 =

0.51 (2)

0.774

Coloured

47

66.2

26

66.7

21

65.6

Black

18

25.3

9

23.1

9

23.1

Other

6

8.5

4

10.3

2

6.3

Marital status

x2 =

= 0.29 (1)

0.591

Never married

51

71.8

27

69.2

24

75.0

Ever married

20

28.2

12

30.8

8

25.0

Highest level of education

x2 =

= 0.15 (2)

0.930

Grade 7 or less

12

17.7

6

16.7

6

18.7

Grades 8 to 11

42

61.8

22

61.1

20

62.5

Grade 12

14

20.6

8

22.2

6

18.8

6 month employment

x2 =

0.31 (1)

0.579

Unemployed

47

69.1

28

71.8

19

65.5

Employed

21

30.9

11

28.2

10

34.5

Diagnosis

0.604

Schizophrenia Spectrum

62

80.5

32

76.2

30

85.7

Bipolar mood disorder

11

14.3

7

16.7

4

11.4

Substance induced psychotic disorder

4

5.2

3

7.1

1

2.9

Substance use

x2 =

= 0.18 (1)

0.671

Lifetime substance use disorder

31

40,3

16

38.1

15

42.9

Antipsychotic

First generation

50

64.9

26

61.9

24

68.6

x2 =

= 0.37 (1)

0.542

Second generation

19

24.7

12

28.6

7

20.0

x2 =

= 0.75 (1)

0.385

Long acting injectable

22

28.6

10

23.8

12

34.29

x2 =

= 1.03 (1)

0.311

Baseline measures

PANSS subscales

Positive

15.4

6.5

15.6

6.9

15.2

6.2

t =

0.06 (73)

0.951

Negative

14.4

4.7

13.8

4.8

15.1

4.5

t =

- 1.47 (73)

0.015

General

26.8

7.5

26.5

7.9

27.1

7.2

t =

- 0.44 (73)

0.663

Total

56.6

15.9

55.9

17.1

57.4

14.5

t =

- 0.42 (73)

0.676

CGI

3.5

1

3.4

1

3.7

1

t =

- 0.96 (68)

0.340

GAF

48.8

10.1

49.9

10.8

47.6

9.4

z =

1.170

0.242

CAN unmet needs

4.1

2.98

3.6

2.97

4.7

2.94

z =

- 1.705

0.088

EUROQUEL VAS

8.4

20.7

84

21.2

84.8

20.4

z =

- 0.200

0.842

MARS

5.9

1.88

5.8

1.87

6.0

1.93

t =

- 0.36 (71)

0.723

a Baseline variables with missing data included:

marital status: n

= 6, ethnicity

: n = 6, HLOE: n = 9, employment = 9, PANSS: n =

: 2, CGI: n

= 7, GAF score: n

= 14, CAN:

n = 6, EUROQUEL VAS: n = 3, MARS: n = 4

Table 3 Efficacy outcomes

Outcome

n

%

Risk ratio (ITT)

p value

95% CI

Unadjusted

(n = 77)

Adjusteda

(n = 77)

Intention-to-treat analysis (ITT): non-adherence to first clinic visit, re-admission over 9 months

Non-adherence to first clinic appointment

Intervention (n = 42)

14

33.3

0.72

0.79

0.419

0.44 to 1.39

Treatment as usual (n = 35)

16

45.7

-

-

-

-

Any re-admission over 9 months

Intervention (n = 42)

5

11.9

0.83

0.86

0.713

0.39 to 1.87

Treatment as usual (n = 35)

5

14.3

-

-

-

-

Outcome             Complete case analysisc (intervention vs. TAU)

ITT (intervention vs. TAU)

Mean difference*1

p value

95% CI

Mean difference*8 (n = 77)

p value

95% CI

Unadjusted

Adjusted

Unadjusted

Adjusted

Complete case and intention to treat analysis (ITT) of other efficacy outcomes at 3 months

PANSS score

Total score           - 9.4

- 14.7

0.052

- 29.71 to 0.16

- 13.4

- 13.1

0.062

- 27.00 to 0.73

Positive subscale - 3.8

- 6.4

0.011

- 11.20 to 1.60

- 5.6

- 5.4

0.060

- 11.16 to 0.25

Negative subscale - 2.6

-4.4

0.059

- 8.99 to 0.18

- 3.5

- 3.5

0.078

- 7.52 to 0.43

General subscale - 2.8

- 3.9

0.350

- 12.6 1 to 4.68

- 4.4

- 4.2

0.248

- 11.67 to 3.19

MARS              - 0.21

- 0.75

0.425

- 2.68 to 1.17

0.36

0.49

0.603

- 1.44 to 2.43

CGI                   - 0.8

- 0.58

0.346

- 1.84 to 0.67

-

-

-

-

GAF                 7.5

4.1

0.440

- 6.90 to 15.17

-

-

-

-

CAN

-

-

-

-

Total needs         -

-

-

-

-

-

-

-

Unmet needs      - 3.2

- 3.6

0.029

- 6.74 to 0.49

-

-

-

-

Met needs         -

-

-

-

-

-

-

-

EUROQUEL-VAS        16.1

15.2

0.124

- 4.59 to 34.99

-

-

-

-

a Log-Poisson regression model with robust variance estimation, ITT; adjusted for age, sex, substance use disorder, baseline scores of PANSS total score, GAF score, MARS, EUROQUEL VAS scale, CAN unmet needs score

b Log-Poisson regression model with robust variance estimation, ITT; sex dropped from model, adjusted for adjusted for age, substance use disorder, baseline scores of PANSS total score, GAF score, MARS, EUROQUEL VAS scale, CAN unmet needs score. Baseline data were imputed

c Multiple linear regression models adjusted for age, sex, substance use disorders, baseline scores of PANSS total score, GAF, EUROQUEL, MARS, CAN unmet needs. Models with violation of linear regression assumptions omitted

d Sample size varied due to list-wise deletion

e Only models for which missing data < 55% are reported

Brief psychoeducation has reduced relapse in the medium term and promoted medication adherence in the short term [31]. Improved clinical outcomes have included reduced relapse and improvement in treatment adherence [32]. Many of our participants reported inadequate and inconsistent current psychoeducation approaches within routine MHSU discharge processes, potentially alluding to inadequate discharge planning. Good discharge planning may be affected by the quality of the therapeutic alliance, which in turn is often impacted by the clinician’s clinical experience [33-36].

While text messaging is a promising and acceptable tool to aid medication and clinic adherence in mental illness, the evidence base remains inconclusive [14, 3738]. Our formative work found support for text message prompts [17]. Some challenges encountered during the trial however, included participant factors such as changing numbers and loss of handsets, impacting negatively on the utility of text messaging, as did fieldworker factors including software challenges and loss of handsets. Socioeconomic factors and the complexity of the software thus impacted negatively on the utility of text message prompts in our setting.

In conclusion, a treatment-partner psychoeducation intervention is acceptable and feasible in a LMIC setting. Psychoeducation must be tailored to the needs of the specific population targeted, and take into account the need for ongoing reinforcement. M-health may have potential to improve adherence, but text message prompts may be problematic in some LMIC settings such as ours at present. Such additional components of such interventions must be tailored to the local context. The assessment of efficacy for such an intervention requires appropriately powered studies.

Limitations

Additional files

Additional file 1. Time frames and associated instruments.

Additional file 2. CONSORT Study flow chart.

Abbreviations

LMIC: low- and middle-income countries; M-health: mobile health; MHSU: mental health service user; CHC: community health centre; VBH: Valkenberg hospital; TAU: treatment as usual; SCID-1: structured clinical interview for diagnosis of axis-I disorders; GAF: global assessment of function scale; CGI: clinical global impressions; PANSS: positive and negative syndrome scale; CAN: Camberwell assessment of needs scale; MARS: medication adherence rating scale; SD: standard deviation; LOCF: last observation carried forward; MAR: missing at random; MICE: multiple imputation with chained equations; VBH: Valkenberg hospital; ITT: intention to treat analysis.

Authors' contributions

DS, PM, ES, and GT conceptualized the original topic for investigation. GS prepared the protocol and all ethics submissions. All authors participated in the development of questionnaires and psychoeducation guides. HT and GS conducted the baseline and 3 month clinical assessments. PW facilitated contact with the mental health nurses. SM and GS conducted the 3-month qualitative interviews, supervised by CL. HT conducted the statistical analyses. GS interpreted the analyses and drafted the manuscript. All authors reviewed the manuscript and approved the final version for submission. All authors read and approved the final manuscript.

Author details

1 Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa. 2 Institute of Psychiatry Psychology and Neuroscience, King's College London, London, UK. 3 Columbia University Mailman School of Public Health, New York State Psychiatric Institute, New York, USA.

Acknowledgements

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The study protocol was approved for all sites by the Human Research Ethics Committee, Faculty of Health Sciences, University of Cape Town (HREC REF: 511/2011). All participants signed informed consent prior to any study procedures. All treatment partners received a handout containing information about the study, informed consent and confidentiality. Anonymity of participants was protected at all discussions by the use of pseudonyms.

Funding

Initial funding for this study was received from the World Psychiatric Association. DS AND GS were supported by the South African Medical Research Council Unit on Anxiety and Stress Disorders. CL was supported by the Programme for Improving Mental health care (PRIME), funded by UK aid from the UK Government, however the views expressed do not necessarily reflect the UK Government's official policies. SM has received support from a National Research Foundation, South Africa postdoctoral fellowship as well as a Harry Crossley Foundation post-doctoral fellowship. PW, HT and PM are specialist psychiatrists in public service with no external funding. GT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King's College London Foundation Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. GT acknowledges financial support from the Department of Health via the National Institute for Health Research (NIHR) Biomedical Research Centre and Dementia Unit awarded to South London and Maudsley NHS Foundation Trust in partnership with King's College London and King's College Hospital NHS Foundation Trust. GT is supported by the European Union Seventh Framework Programme (FP7/2007-2013) Emerald project. ES's in-kind contribution was supported by New York State Psychiatric Institute, New York.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Received: 11 August 2017 Accepted: 3 November 2017

Published online: 09 November 2017

References

Submit your next manuscript to BioMed Central and we will help you at every step:

Submit your manuscript at

www.biomedcentral.com/submit                     Central

FILIP SPANIEL, MD JAN HRDLICKA, MA TOMAS NOVAK, MD JIRI KOZENY, PsyD CYRIL HOSCHL, MD PAVEL MOHR, MD LUCIE BANKOVSKA MOTLOVA, MD


Effectiveness of the Information Technology-Aided Program of Relapse Prevention in Schizophrenia (ITAREPS): A Randomized, Controlled, Double-Blind Study

Purpose. To evaluate the effectiveness of the Information Technology-Aided Program of Relapse Prevention in Schizophrenia (ITAREPS). Methods. Relapse-prone outpatients with schizophrenia or schizoaffective disorder were randomized to the active (n = 75) or control group (n = 71). In the active arm, according to the protocol, investigators were prompted to increase the antipsychotic dose upon occurrence of a pharmacological intervention requiring event (PIRE) detected by ITAREPS. Results. Intention-to-treat (ITT) analysis found no between-group difference in the hospitalization-free survival rate at 12 months. However, the trial suffered from high non-adherence of investigators in the active group, with no antipsychotic dose increase in 61% of PIREs. Furthermore, Cox regression analysis showed a 11-fold increased risk of hospitalization in the absence of pharmacological intervention following a PIRE (hazard ratio [HR] = 10.8; 95% confidence interval [CI] 1.4-80.0; p = 0.002). Therefore, a post-hoc as-treated analysis was performed, which demonstrated a nine-fold reduction in the risk of hospitalization in ITAREPS Algorithm-Adherers (IAAs, n = 25) compared with the ITAREPS Non-intervention-al group (INIs, n = 70; Kaplan-Meier survival analysis, HR = 0.11, 95% CI 0.05-0.28, p = 0.009; number needed to treat [NNT] = 4, 95% CI 3-10). A significant difference in favor of the IAA group was seen in the number of inpatient days (p < 0.05) and costs (p < 0.05). Conclusion. Future ITAREPS trials should target the underlying mechanisms that cause low investigator adherence to the program. Trial registration: Clinical-Trials NCT00712660 (Journal of Psychiatric Practice 2012;18:269-280)

KEY WORDS: schizophrenia, psychotic disorders, relapse prevention, hospitalizations, antipsychotic medication, information technology

The annual relapse rate in schizophrenia, usually indicated by psychiatric hospitalization, is relatively high, varying from 35% for patients who are adherent to treatment to approximately 75% for poorly adherent individuals.1,2 Recurrent psychotic episodes appear to have a serious detrimental impact on clinical outcome, including social decline and diminished treatment response.3-5 In addition, it has been suggested that the “toxic” effect of the psychotic state contributes to the structural brain abnormalities found in schizophrenia.6-10

Rehospitalization imposes a substantial economic burden on health care systems, representing approximately 85%-95% of the costs of schizophrenia relapse.11 Although the proportion of health care costs attributed to inpatient care in schizophrenia may vary from country to country, inpatient hospital-

SPANIEL, NOVAK, KOZENY, HOSCHL, MOHR, and BANKOVSKA MOTLOVA: Prague Psychiatric Center, 3rd Faculty of Medicine, Charles University, Center of Neuropsychiatric Studies, Prague, Czech Republic; HRDLICKA: Czech Technical University, Faculty of Electrical Engineering, Department of Cybernetics, Prague.

Copyright ©2012 Lippincott Williams & Wilkins Inc.

Please send correspondence to: Filip Spaniel, MD, Prague Psychiatric Center, Ustavni 191, 181 03 Prague 8, Czech Republic. spaniel@pcp.lf3.cuni.cz

Acknowledgement: This was an investigator-initiated trial funded by Eli Lilly Company. Eli Lilly had no further role in the collection, analysis, and interpretation of data. ITAREPS was developed by the Prague Psychiatric Centre as an academic project, technically supported by Academia Medica Pragensis (Amepra). Amepra is an agent and ethical guarantor of relations between Eli Lilly Czech Republic which provided funds for ITAREPS development and the academic group involved in the project development and implementation. ITAREPS (www.itareps.com) is operated under the auspices of the Psychiatric Society CLS JEP of the Czech Republic and the Psychiatric Society SLS of the Slovak Republic. We greatly appreciate the consultation and invaluable help in study design of Drs. R. Brousil and P. Vohlidka.

Filip Spaniel MD, Tomas Novak MD, Jiri Kozeny, PsyD, Cyril Hoschl, MD, Pavel Mohr, MD, and Lucie Bankovska Motlova, MD, received honoraria from Eli Lilly Co. Jan Hrdlicka, MA, declares no conflicts of interest.

DOI: 10.1097/01.pra.0000416017.45591.c1 ization is the largest contributor to the overall direct costs of treating schizophrenia.1213

Given the disruptive effects of relapses on the lives of patients with schizophrenia and the high cost of inpatient treatment, tertiary prevention is a major goal of long-term treatment. This aim seems to be achievable since, in the majority of cases, relapse is heralded by early warning signs.14 A reduction in the risk of rehospitalization via timely pharmacological intervention in the prodromal stage has previously been demonstrated.15 Such evidence led the Prague Psychiatric Center to develop the early detection and intervention Information Technology-Aided Program of Relapse Prevention in Schizophrenia (ITAREPS).16 A previous mirror-design study indicated that early warning signs of relapse can be identified by ITAREPS and that relapses can be aborted and the hospitalization rate reduced by an appropriate increase in antipsychotic dose within the detected prodromal period.17

In this article, we present the results of an international multicenter trial of the ITAREPS program. The study involved a 1-year prospective, parallel-group, randomized, double-blind controlled trial carried out at 35 outpatient psychiatric centers in the Czech Republic (29 sites) and in the Slovak Republic (6 sites). Investigators were psychiatrists from state, county, and private outpatient facilities.

The study was designed to test the effectiveness of early pharmacological intervention upon occurrence of early warning signs detected by ITAREPS in outpatients with schizophrenia or schizoaffective disorder who were at a high risk of relapse. Other psychosocial and supportive interventions were not the focus of the trial. The primary endpoint of this trial was the difference between the active and control groups in the 12-month survival without hospitalization due to relapse of psychosis. The secondary endpoints were differences in a) the number of inpatient days, b) direct inpatient costs between the active and the control groups and c) identification of predictors of hospitalization due to relapse.

SUBJECTS AND METHODS

Participants

The study subjects were men and women 18-60 years of age and their healthy family members. The inclusion criteria were

The exclusion criteria were:

The trial protocol was reviewed and approved by local independent ethics committees according to the regulatory requirements in both participating countries. Informed consent was obtained from each patient and family member before enrolment in the study.

Study Design

Patients and family members were to complete the 10-item Early Warning Signs Questionnaire16 (EWSQ-10P patient version and EWSQ-10FM family version, respectively) upon receiving a short message service (SMS) request sent automatically once a week by the ITAREPS computer system to their mobile phones. Scores on each of the 10 items ranged from 0 (no worsening or even improvement of symptoms) to 4 (extreme worsening). All participants then sent their individual EWSQ scores back to the ITAREPS system as a 10-digit SMS message. ITAREPS automatically detected proportional worsening (or a new onset) of prodromal symptoms compared with the previous week's baseline. If a total EWSQ score exceeded a preset score threshold, an automatically generated ALERT e-mail was sent to the treating psychiatrist with a prompt to follow the “per protocol” intervention.

After screening, a 3-week training phase followed, during which all study subjects completed their EWSQs, but no alert announcements were delivered to investigators. After the training period, patients were randomized in a 1:1 ratio into the active and control groups via a centralized computer-based dynamic random allocation method in the remote ITAREPS central database in order to balance the treatment groups across the following stratification variables: age, gender, duration of illness, CGI score, Hayward Medication Compliance Scale score, number of previous hospitalizations, current medication (oral or depot), level of education, and score on the Global Assessment of Functioning (GAF) scale.21

After randomization, both groups of patient/family member pairs continued to receive weekly ITAREPS requests for EWSQ scores and to provide their EWSQ results to the system on a regular basis for the 12 months of the study.

In the active group, ITAREPS reported the occurrence of prodromes to the investigators via ALERT emails. This alerting service was completely disabled in the control group. Therefore, in the control group, the investigators detected and reacted to potential signs of psychotic relapse as they would have done prior to enrolment in the study. In addition to protocol-directed intervention in the active group, all patients received routine clinical and medication management with a frequency of visits that was usual in the outpatient clinical settings in which they were being treated.

Study subjects and investigators were blind to randomization status and study design (i.e., whether it was parallel or multiple/cross-over). As the investigators were not sure whether they would receive any feedback from ITAREPS, they had to rely on clinical judgment in the case of worsening of a patient's condition, using regular in-label antipsychotic treatment of their own choice with appropriate dose adjustments according to clinical need.

When the EWSQ score from the patient and/or the respective family member in the active group exceeded the predefined default threshold, an INITIAL ALERT (IA) e-mail was sent to the investigator. This IA was followed by a 3-week ALERT PERIOD (AP), during which subjects were prompted by ITAREPS to return the EWSQs twice weekly and a more conservative score threshold was used. The AP was terminated when six consecutive EWSQ scores showed no further worsening of symptoms. If EWSQ scores exceeded the more conservative threshold at any time during the AP, an ALERT EMERGENCY (AE) was announced to the investigator via e-mail. The AP was also extended for a further 3 weeks after each new AE message. Both the IA and AE thresholds were previously determined to maximize the hospitalization predictive value by evaluating pooled patient data that had been collected in the ITAREPS database since the introduction of the program in clinical practice in 2005.

The core ITAREPS study intervention was a “per protocol” 20% increase in the dose of antipsychotic medication within 24 hours in response to a Pharmacological Intervention Requiring Event (PIRE). A PIRE was defined as either:

Seventy-two hours following each IA or AE announcement, investigators were prompted via email to complete an unscheduled post-ALERT visit, to assess the clinical change by means of the Clinical Global Impressions-Improvement Scale (CGI-I) (proportional change compared to the last contact before announcement of the current IA or AE), and to record changes in antipsychotic medication (if any) so that the exact nature of the intervention and the investigator's adherence to the protocol could be documented.

Three study visits were scheduled after screening: at baseline, month 6, and month 12 (end-of-study visit). All collected data were web-based and investigators were prompted to perform data input via an automatic e-mail containing a direct link to the web page with the relevant case report form.

Measures

Study assessments included the CGI-S and CGI-I scales,19 the Hayward 7-item Medication Compliance Rating Scale,20 and the GAF scale21 at baseline, month 6, and the end-of-study visit. Demographic data and the history of illness were obtained at baseline. CGI scores, dates of hospitalization, dose of antipsychotic medication and any changes, hospitalization due to relapse, and also relapse without subsequent hospitalization were recorded at each outpatient appointment. Relapse without hospitalization was defined as a CGI-I > 6, self-injury, suici-dal/homicidal ideation, or violent behavior.22

Statistical Analyses

In order to detect a 20% difference in the 12-month hospitalization-free survival with a one-sided type I error of 5% and a power of 90%, we estimated that 150 patients had to be enrolled in the study (75 per group). For power analysis, we used data from the previous ITAREPS mirror-design study.17

In the intention-to-treat and as-treated analyses, we estimated the cumulative probability of remaining free of hospitalization due to relapse using the Kaplan-Meier method and compared this variable with log-rank statistics. An overall risk difference and risk ratios were calculated at the end of the study with 95% confidence intervals.

We conducted an intention-to-treat analysis that included all randomized patients. Post-hoc analyses (as-treated analysis and Cox proportional hazards regression) were performed to examine the effect of the specific ITAREPS-mediated intervention (i.e., antipsychotic dose increase upon announcement of a PIRE). Therefore, subjects with no prodromal signs detected by ITAREPS during the course of the study (prodrome negatives, PNs) were excluded from these analyses. An as-treated analysis was carried out with stratification by the PIRE+ variable. PIRE+ was defined as the proportion of PIREs followed by the “per protocol” required antipsychotic dose increase out of the total number of PIREs released for the given patient during the study. The stratification by PIRE+ was implemented irrespective of the patients' assignment to either the active or the control group.

Post-hoc Cox proportional hazard regression analysis was used to estimate the effects of multiple potential risk factors for hospitalization due to relapse in the whole sample during the 1-year study. The following independent variables were used: age at baseline, gender, number of hospitalizations and duration of illness, level of education, age at the onset of illness, baseline CGI-S score, diagnosis (schizophrenia or schizoaffective disorder), baseline Hayward Medication Compliance Scale score, baseline GAF score, formulation of medication at entry into the study (oral or depot), patient/family member adherence with ITAREPS (for definition see below), and whether the antipsychotic dose was appropriately increased according to the protocol (PIRE+). We used 95% confidence intervals to indicate the precision of the hazard ratios obtained via these analyses.

Clinical characteristics, number of days hospitalized, length of prodromal symptom periods, and direct costs of treatment were compared in univariate analyses using the Mann-Whitney U test for continuous variables and the chi-square test for categories. The costs of psychiatric hospitalization were based on direct daily costs covered in 2009 by Czech health insurance companies. Cost data were converted to Euros (EUR).

RESULTS

Study population

We screened 442 patients between April, 2008, and February 2009 (Figure 1). In 251 cases, either the patient or the family member refused to participate in the study, and an additional 33 patients were excluded because they did not meet inclusion criteria. Thus, 158 patients were randomly assigned to one of two study groups. Twelve of these subjects withdrew consent early in the study, so that the dropout rate due to withdrawal of consent was 7.6% per year (5% in the active and 10.1% in the control group, respectively). These 12 patients withdrew before any prodromal symptoms were detected by ITAREPS. Therefore, the total number of patients in the study (intention-to-treat population) was 146 (75 in the active and 71 in the control group). Both groups were similar in terms of baseline variables (Table 1). In total, 25 of the 35 investigators had both


active and control patients in their registries

During the 1-year study, participants sent a total of 17,082 SMSs (8741 in the active and 8341 in the control group) in response to 21,208 automatically generated ITAREPS prompts. Overall, the return rate was 80% (active = 79.8%, controls = 81.3%). In the active group, ITAREPS detected and announced 1,146 alerts (either INITIAL ALERTS or ALERT EMERGENCIES), representing 13.1% of all SMS messages received in this study arm. Of these 1,146 alerts, 149 required immediate pharmacological intervention according to the study protocol and thus fell into the category of a pharmacological intervention requiring event (PIRE). In the control group, ITAREPS detected 1,911 alert events representing 22.9% of all SMS messages received.

Adherence to the Protocol

Subject adherence to ITAREPS. Thirty subjects (20.5% of all those who were randomized; 13.3% in the active and 28.2% in the control group) met predefined criteria for non-adherence to ITAREPS (i.e. patient/family member responded to fewer than 70% of SMS prompts).

Investigator protocol adherence. The adherence of investigators to the protocol was poor in the active group. Despite protocol-directed therapy, the antipsychotic dose was increased in response to only 58  (39%) of 149 PIREs.

Investigators reported the following reasons for their failure to perform the required pharmacological intervention in the case of the other PIREs (n = 91): clinical inappropriateness of dose increase according to investigator's judgment (64.9% of all cases), patient's refusal to comply with higher doses (15.5%), inability to contact the patient (6.2%), inability to increase the dose due

to medication side effects (6.2%), inability to increase the dose since the maximum recommended dose had already been reached (4.1%), and risk resulting from rapid dose increase in the case of consecutive announcements of PIREs in the given patient (3.1%).

ITAREPS identified early warning signs of relapse at least once during the course of the study in 67% of active patients (Active Prodrome Positives; APP, n = 50), whereas 33% of the active patients remained free of early warning signs (Active Prodrome Negatives; APN, n = 25). The system identified a similar proportion (63%) of patients with early warning signs in the control group (Control Prodrome Positives; CPP, n = 45). However, more alerts were issued for individual patients in the control group, accounting for the higher number of alerts overall in that group (1911 versus 1146 in the active group). The prevalence of the investigators' failure to increase the antipsychotic dose in response to a PIRE (at least one omission to increase dose according to the protocol occurred in 44% of active patients) meant that APP patients varied in terms of the PIRE+ ratio (i.e., proportion of PIREs followed by a dose increase out of the total number of PIREs for a given patient).

For post-hoc analyses, we arbitrarily divided the APP patients into two categories: ITAREPS Algorithm-Adherers (IAAs, n = 25 for whom more than 50% of PIREs were followed by the required dose increase), and ITAREPS Algorithm-Neglectors (IANs, n = 25 for whom fewer than 50% of PIREs were followed by the required dose increase). The rationale behind this “all-or-nothing” categorization was the fact that 83% of prodrome-positive subjects from the pooled study sample followed a dichotomous distribution of PIRE+ values (i.e., the “per protocol” required pharmacological intervention was carried out in either 100% or 0% of all PIREs released for those patients during the study). Obviously, the PIRE+ ratio in prodrome-positive controls was always zero. In the remaining 17% of active patients (n = 16), in whom the PIRE+ varied from 14% to 75%, a mid-distribution dichotomous 50% cut-off point for either “ITAREPS AlgorithmAdherers” (IAAs) or “ITAREPS Non-Interventional individuals” (INIs) allocation was used, since an equal number of patients (8 subjects) fell either below or above this value. However, we are fully aware that one of the main study flaws results from the fact that allocation to the IAA and INI groups was not predefined in the study protocol.

A logistic regression model with ”investigator adherence” as dependent variable did not show any relation between study site or clinical and patient demographic variables and investigator non-adher-ence within the active group. In addition, no differences in clinical and demographical variables were found between IAAs and IANs.

Primary Endpoint: Hospitalization-Free Survival Rate

Intention-to-treat analysis. In the ITT population (n = 146), 13 of the 75 patients in the active group (17.3%) were hospitalized due to relapse of psychosis compared with 12 of the 71 patients (16.9%) in the control group. No statistically significant differences were found in patient survival between the active and the control groups using the Kaplan-Meier method plus the log-rank test (hazard ratio [HR] 1.05; 95% confidence interval [CI] 0.48-2.30; p = 0.9, Figure 2A).

As-treated analysis. In the post-hoc as-treated analysis, we focused on the primary outcome according to the intervention actually received. In order to assess the direct preventive effect of the ITAREPS-triggered pharmacological intervention, only subjects from both the active and the control groups with early warning signs detected by the system (n = 95) were included, while all prodrome-negative subjects from both arms (n = 51) were excluded from the analysis. In this analysis, ITAREPS Algorithm-Adherers (IAAs, = 25) were compared with a virtual control group consisting of ITAREPS Algorithm-Neglectors (IANs, n = 25) and Control Prodrome-Positives (CPPs, n = 45). IANs + CPPs constituted a pooled virtual ITAREPS Non-Interventional group (INIs, n = 70).

Kaplan-Meier survival estimation using the logrank test showed statistically significant differences in patient survival according to therapy, with significantly better hospitalization-free survival for IAAs (1 of 25 patients hospitalized [4%]) compared with the INI group (22 of 70 hospitalized [31%]; HR 0.11, 95% CI 0.05-0.28, p = 0.009; number needed to treat [NNT] = 4, 95% CI 3-10) (Figure 2B).

Secondary endpoints

Inpatient days and direct inpatient costs. There were no differences in the number of inpatient days and direct inpatient costs (EUR) in the intention-to-

treat analysis (Table 2). However, the as-treated analysis found a mean of 2.1 (standard deviation [SD] 10.2) inpatient days for the IAAs subjects compared with a mean of 19.7 days (SD 38.4) for the INI subjects (Mann-Whitney U test, Z = -2, p < 0.05). The analysis also demonstrated a statistically significant difference in direct inpatient costs in EUR favoring the IAA group (mean 81.6 EUR, SD 339.6) over the INI group (mean

Predictors of hospitalization due to relapse. A post-hoc Cox proportional hazards regression analysis was carried out to determine independent predictors of hospitalization due to relapse during the 1-year study. This analysis revealed that the absence of pharmacological intervention following a PIRE was significantly associated with the risk of hospitalization (HR = 10.8; 95% CI 1.4-80.0; p = 0.002). The other risk factors were male gender (HR = 3.6, 95% CI 1.2-10.5, p = 0.007) and lower education (HR = 1.3, 95% CI 1.03-1.6, p = 0.049).

DISCUSSION

This study was an investigator-initiated multicenter trial using the ITAREPS program to prevent relapse in schizophrenia. In the primary intention-to-treat analysis, our results showed no difference in the hospitalization-free survival rate between the active and the control groups. However, this was largely a failed study, because investigators adhered to the protocol by increasing the antipsychotic dose in only a minority (39%) of PIREs.

Although the trial could not achieve its stated goals, we considered it important to report our


results for several reasons. First, our findings underline the difficulties that may arise due to the indistinct phenomenology of early warning signs. The most common reason (65%) given by investigators for not responding to the PIREs was that, based on clinician's judgment, the severity of the early warning signs did not justify prompt pharmacological intervention, even though failure to increase the antipsychotic dose required deviation from the protocol. The literature on the nature of prodromal symptoms suggests that early warning signs of relapse are subtle, with generally very small mean elevations of objective measures of psychopathology.23-26 In this study, the median CGI-I score in all PIREs was 5, reflecting only minimal worsening of the clinical status. A dose increase of 20% is clinically significant, especially in light of the absence of substantial decline in symptomatology and thus questionably acceptable to treating psychiatrists. This factor plausibly undermined investigators' adherence to the dose increase requirement. To sum up, psychiatrists are probably generally reluctant to react pharmacologically to such minor subclinical signs of relapse because they prefer to rely on their clinical instinct. This was also supported by the fact that PIREs followed by pharmacological intervention had significantly higher CGI-I scores (mean 5.09, SD 0.65) compared with PIREs with no intervention (mean 4.57, SD 1.09, p = 0.00005, Z = 4.07, Mann-Whitney U test).

Second, unexpectedly high investigator non-adher-ence to the protocol in the active arm affected the results of the Cox regression model. The strongest predictor of hospitalization was the absence of pharmacological intervention after ITAREPS prodrome detection (PIRE). There was an 11-fold increase in the hospitalization risk in patients in the whole prodrome-positive sample who received no antipsychotic dose increase in response to PIREs. The analysis also showed a protective effect of female gender and of higher education against the risk of inpatient admission, which is consistent with previous reports.27,28

To assess the effectiveness of the core ITAREPS “per protocol” pharmacological intervention, an exploratory post-hoc as-treated analysis was carried out in the sample of patients in whom early warning signs were detected by the system. This showed a significant effect of patient stratification according to the presence or absence of an antipsychotic dose increase in response to the ITAREPS-identified prodromes classified as PIRE. As-treated analysis comparing the IAAs and the INIs demonstrated that an antipsychotic dose increase upon receipt of the ITAREPS PIRE significantly reduced the risk of hospitalization due to relapse, and decreased the number of inpatient days and the direct costs of inpatient care (secondary outcomes). However, the post-hoc as-treated approach represents a substantial limitation of the trial, compromising the generalizability of the findings. Therefore, the results should be considered indicative rather than conclusive.

Despite antipsychotic dose increases in the IAA group, no significant differences were seen between the IAA and INI groups in the mean daily outpatient antipsychotic medication costs nor the mean daily outpatient antipsychotic dose in chlorpromazine equivalents29 (Table 2), probably due to the complex dynamic of dose manipulations. Relatively frequent dose adjustments were made in response to ITAREPS alerts in the IAA group, although those episodes were relatively short. Concurrently, in the INI group, there were few but prolonged episodes of dose increases that typically followed previously ignored alerts. This was probably in response to an apparent progressive decline in clinical state. Thus, those different approaches may have led to a similar chlorpromazine equivalent dose load in both groups.

In summary, intention-to-treat analysis did not yield a significant difference between treatment groups defined by the protocol in this trial. The study showed low psychiatrist adherence with the ITAREPS algorithm, which we believe reflects psychiatrists' lack of confidence in warning methods based on rating scales and information technology. In this respect, we speculate that investigators did not want to surrender their authority and treatment to an impersonal machine-like procedure. However, this conclusion remains speculative, since qualitative data on investigators' and users' perception of the ITAREPS were not available in this study. Nevertheless, the findings taken together provide an incentive for further development of the system. The predictive power of the EWSQ score thresholds has recently been substantially improved, using a cybernetic, machine learning approach, so that the number of false negative alerts has been reduced by 40%. Based on the experience we have gained, we are currently controlling for an action taken after an alert delivery in a new version of the program, ITAREPS 2.0, in which psychiatrists are instructed to announce electronically (upon automatic email prompt) the type of pharmacological intervention or the reason for absence of dose increase. Since the study presented here showed an absence of detailed information on baseline symptoms due to exclusive use of the CGI to characterize the sample, two ITAREPS studies that are now underway are using the Positive and Negative Syndrome Scale (PANSS)30 to better evaluate the feasibility of the method within heterogeneous patient populations.

This study also suggested innovative ways of further exploiting this telemedicine technology. We can see promising potential for the program in the evaluation of long-term clinical characteristics of different antipsychotic medications as measured by means of EWSQ fluctuations as a proxy for clinical stability. Therefore, ITAREPS may also have a role in the provision of personalized treatment. Implementation of the ITAREPS in the context of intensive case management would also be fruitful, although it was not possible to investigate that question in this study given the lack of those services in the Czech Republic. We would also expect to find that a combined approach that employed both pharmacological and psychosocial measures in relapse prevention would be superior to either strategy alone. These, among other issues, will be addressed in future studies.

Our goal in this study was to evaluate the feasibility of using a simple preventive algorithm based on antipsychotic dose modifications, to be administered within the common constraints imposed by an outpatient treatment milieu. Our findings indicate that implementing this novel approach to relapse prevention in clinical practice will remain challenging unless validated computerized methods gain acceptance and a substantial change in clinical thinking is achieved.

CONCLUSION

Our results suggest that a relapse prevention program based on telecommunication technologies may represent a novel, cost-effective, and simple method for reducing disease burden and direct inpatient costs in schizophrenia. The study presented here provided a framework for further investigation of a refined concept of the ITAREPS system.

References

BMJ Open Smartphone-delivered self-management for first-episode psychosis: the ARIES feasibility randomised controlled trial


Thomas Steare   ,1 Puffin O’Hanlon,1 Michelle Eskinazi,1 David Osborn,1,2

Brynmor Lloyd-Evans,1,2 Rebecca Jones,1 Helen Rostill,3,4 Sarah Amani,5 Sonia Johnson 1,2


To cite: Steare T, O'Hanlon P, Eskinazi M, etal. Smartphone-delivered self-management for first-episode psychosis: the ARIES feasibility randomised controlled trial. BMJ Open 2020;10:e034927. doi:10.1136/ bmjopen-2019-034927

► Prepublication history and additional material for this paper are available online. To view these files, please visit the journal online (http://dx.doi. org/10.1136/bmjopen-2019-034927).


Received 11 October 2019 Revised 27 June 2020 Accepted 02 July 2020


Check for updates


© Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY. Published by BMJ. 1Division of Psychiatry, University College London, London, UK 2R&D Department, Camden and Islington NHS Foundation Trust, London, UK 3University of Surrey, Guildford, UK

4Surrey and Borders Partnership NHS Foundation Trust, Leatherhead, UK 5Early Intervention in Psychosis Programme (South of England), Oxford, UK


ABSTRACT

Objectives To test the feasibility and acceptability of a randomised controlled trial (RCT) to evaluate a Smartphone-based self-management tool in Early Intervention in Psychosis (EIP) services.

Design A two-arm unblinded feasibility RCT. Setting Six NHS EIP services in England.

Participants Adults using EIP services who own an Android Smartphone. Participants were recruited until the recruitment target was met (n=40).

Interventions Participants were randomised with a 1:1 allocation to one of two conditions: (1) treatment as usual from EIP services (TAU) or (2) TAU plus access to My Journey 3 on their own Smartphone. My Journey 3 features a range of self-management components including access to digital recovery and relapse prevention plans, medication tracking and symptom monitoring. My Journey 3 use was at the users' discretion and was supported by EIP service clinicians. Participants had access for a median of 38.1 weeks.

Primary and secondary outcome measures Feasibility outcomes included recruitment, follow-up rates and intervention engagement. Participant data on mental health outcomes were collected from clinical records and from research assessments at baseline, 4 months and 12 months.

Results 83% and 75% of participants were retained in the trial at the 4-month and 12-month assessments. All treatment group participants had access to My Journey 3 during the trial, but technical difficulties caused delays in ensuring timely access to the intervention. The median number of My Journey 3 uses was 16.5 (IQR 8.5 to 23) and median total minutes spent using My Journey 3 was 26.8 (IQR 18.3 to 57.3). No serious adverse events were reported.

Conclusions Recruitment and retention were feasible. Within a trial context, My Journey 3 could be successfully delivered to adults using EIP services, but with relatively low usage rates. Further evaluation of the intervention in a larger trial may be warranted, but should include attention to implementation.

Trial registration ISRCTN10004994.

Correspondence to

Prof Sonia Johnson;

s.johnson@ucl.ac.uk


INTRODUCTION

Early Intervention in Psychosis (EIP) services have been established across the UK to

Strengths and limitations of this study

provide care to adults during the 3 years following an initial episode of psychosis. There is evidence that such services are effective and cost-effective,1 2 resulting in improvement in a range of outcomes, yet challenges remain. Relapse rates for EIP service users are high3 particularly after discharge4 5 and limited adherence with antipsychotic medication is common.6 There are also difficulties accessing psychosocial interventions,7 including supported self-management.

Illness self-management is an approach designed to support people to manage longterm health conditions by developing their ability to recognise and monitor symptoms and early warning signs of relapse, identify and avoid stressors, make plans for achieving their own recovery and effectively use coping strategies.8 For people with psychosis, selfmanagement tools have been shown to reduce psychological distress, improve medication adherence and reduce the likelihood of future hospital admissions.9-11 In a recent meta-analysis, self-management interventions for severe mental illness were also found to have a significant benefit on patient-valued

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


outcomes of personal recovery, hope and self-efficacy.12 Despite clinician-supported self-management programmes being mandated in current UK treatment guidelines for first-episode psychosis,13 there is a lack of well-evaluated tools to support delivery within EIP services. There is a clear need to overcome implementation barriers affecting the delivery of self-management to those likely to benefit from it.12 A potentially convenient and economical way of achieving this is via the use of digital technology such as Smartphones.14

Smartphones can run advanced software known as apps that hold promise as an effective tool to assist the monitoring and treatment of mental health problems. Smartphone ownership is rapidly growing worldwide15 with a significant number of developed countries with ownership rates of more than 80%.16 Adults with severe mental health problems have comparable Smartphone ownership rates with the general population,17-19 and there is a growing consensus that adults with psychosis are open to using Smartphones to access mental health interventions.20 21 Smartphones also provide high accessibility to the internet and are commonly carried on the person, meaning apps can be easily accessed at times and locations convenient for the user. Accordingly, Smartphones have the capacity to deliver time-unlimited mental health interventions, such as self-management tools, and ultimately the potential to increase access to effective care and reduce healthcare costs.22 The benefits of Smartphone apps may also extend beyond the original treatment period with a community team and could be a valuable tool following discharge where the risk of relapse is increased.4 5

The majority of digital health interventions that have been developed for psychosis have been based on existing psychological therapies such as cognitive-behavioural therapy,23 24 or other evidence-based interventions,25 26 yet very little is known regarding their effectiveness when delivered in EIP services. A growing number of selfmanagement apps for psychosis have been tested for feasibility and acceptability, including those delivered independently of a clinical setting and those embedded within clinical care.27-29 These have shown promising levels of adoption and use in research contexts, yet little is known about their clinical efficacy.

To date, only one trial of a self-management app delivered in EIP services has published results regarding the intervention's impact on clinical outcomes.30 In the proof-of-concept trial, an active self-management app ‘Actissist’ was found to confer benefits over a passive control app. The study suggests that participants who received Actissist had better outcomes regarding their mood and general and negative symptoms post-treatment in comparison with control participants. Actissist features a range of components including self-assessment questions focused on cognitive appraisals, emotions, behaviours and belief convictions and suggests appropriate coping strategies, but does not feature some major cornerstones of self-management such as relapse and recovery plans. Regardless, results from this study suggest that such digital self-management interventions could potentially improve outcomes of people using EIP services. Further trials are needed before firm conclusions can be made regarding the feasibility of conducting randomised controlled trials (RCTs) in this field and of the therapeutic benefits of selfmanagement apps for first-episode psychosis delivered in clinical settings.

We aimed to address this evidence gap by conducting a feasibility RCT of a supported self-management Smartphone app, ‘My Journey 3’, designed to help EIP service users recognise early warning signs of illness, recognise and monitor symptoms, and create plans for their recovery. My Journey 3 has been designed to be initially set up in EIP services and used with clinician support, but to also be suitable for independent use. The results of the feasibility RCT are a potential step towards a full-scale trial to assess the effectiveness of the intervention.

The objectives of this study were as follows:

METHODS

Design

The App to support Recovery In Early intervention Services (ARIES) study was an unblinded feasibility RCT with a nested qualitative study comparing a supported self-management Smartphone app (My Journey 3) in addition to treatment as usual (TAU), with a control group receiving TAU only. Participants were randomly allocated to one of the two trial arms in a 1:1 ratio. Since this was a feasibility trial, it was not designed to have sufficient statistical power to assess the effectiveness of the My Journey 3 intervention.

As the study was a feasibility trial, prospective registration was not required.31 Further details of the methodology are available in the protocol paper.32 We have followed the Consolidated Standards of Reporting Trials (CONSORT) statement extension for pilot and feasibility randomised trials for reporting.33 A copy of the CONSORT checklist is provided as online additional file

Setting

The trial was conducted in six EIP services across three NHS Foundation Trusts in England. EIP services are multidisciplinary community mental health services that provide care coordination to people in the first 3 years of a first-episode psychosis, focusing on engagement, achieving social and clinical recovery and delivering a full range of pharmacological, psychological and social interventions.34 The six EIP services as mandated in

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


England provide care for people up to the age of 65, with the potential for adults above the age range to access EIP services if clinically appropriate although these cases are rare. Two of the participating Trusts are located in inner London. The third Trust is located in a county outside of London with both urban and rural areas. Assessments were conducted face-to-face at EIP services, at participants’ homes or at University College London.

Participants

Participants were recruited from the participating EIP services over 7months. We assumed a conservative 40% attrition rate and accordingly set the target sample size as 40 participants to ensure the trial retained 12 completer participants per group (as recommended to assess trial feasibility).35 Participants were eligible if they were aged >16 years, had experienced at least one episode of psychosis, were currently on the caseload of an EIP service and owned a Smartphone with an Android operating system. People were excluded from the trial if they lacked capacity to consent to participation, were unable to communicate and understand English, or were considered by their EIP service to pose a high risk to researchers during meetings, even on NHS premises. Familiarity and competence in using digital technology or Smartphones was not an eligibility criterion.

Recruitment strategy

Clinicians at the participating EIP services were briefed by the research team and were asked to make initial contact with eligible EIP service users. Clinicians explained the trial to service users and enquired whether the service user would be willing to speak to a researcher about participating in the trial. The researcher then made contact with eligible and potentially willing service users and arranged a face-to-face meeting where the trial was explained further. The researcher provided the trial information sheet (online additional file 2) and assessed the participant’s capacity to provide informed consent. Service users had at least 24 hours after receiving the information sheet to consider their participation. Participants then gave written informed consent to take part, prior to completing the baseline assessment. No participants were recruited via online methods.

Randomisation

Following the baseline assessment, participants were randomly allocated in a 1:1 ratio to either the intervention (n=20) or the control group (n=20) by an independent statistician. The treatment group had access to My Journey 3 in addition to TAU, while the control group received TAU only. An independent researcher held the allocation list and did not disclose participants’ allocation to the trial researcher until after completion of the baseline assessments.

Due to the nature of the intervention, participants were not blinded to their group allocation. During the recruitment process, participants would have been aware that My Journey 3 was the intervention of interest. As a single researcher carried out the majority of data collection, it was not practical for the allocation of participants to be concealed from the research team. Participants were informed of their allocation by the researcher via a telephone call.

Interventions

My Journey 3

My Journey 3 is a Smartphone app developed for adults accessing EIP services. The aim of the intervention is to develop users’ self-management skills to help them to achieve self-determined recovery goals and avoid future relapses. My Journey 3 is suitable for independent use, but also designed to be used with support from EIP service clinicians who will be able to assist with the completion of the self-management components and initial set-up. It is the developers’ aspiration for My Journey 3 to be used initially in collaboration with EIP service clinicians, and for it to support continuing self-management after users have been discharged from EIP services.

The development of My Journey 3 has been through several iterations. The first version (My Journey 1) was created by Surrey and Borders Partnership NHS Foundation Trust with leadership from Sarah Amani, for EIP service users to track their symptoms, set reminders for appointments and share their progress with EIP service clinicians. In developing the current version of My Journey 3, we have drawn on existing paper-and-pen self-management intervention components36 37 to allow users to track recovery goals and personalise relapse prevention plans—important cornerstones of illness selfmanagement. The design and the content of My Journey 3 was led by a collaboration of researchers, digital health experts, EIP service clinicians and service users. A private app development company based in the UK (MyOxygen; https://myoxygen.uk) led the technical development of My Journey 3. To limit costs, My Journey 3 is only compatible with Smartphones with Android operating systems at this stage of testing.

My Journey 3 features four key elements of selfmanagement, an approach with demonstrated efficacy in improving social and clinical outcomes for people with psychosis.12 Screenshots of the key components are displayed in figure 1. Users can create a relapse prevention plan, where there is the opportunity to identify and list triggers, early warning signs of relapse and personalised coping strategies to refer to as required and to create a plan to follow if experiencing a crisis. Via the ‘My Recovery Plan’ section, users are able to set recovery goals, list actions they can do to encourage well-being and set reminders on their Smartphone to encourage engagement in these activities. Users can also use a tracker to monitor and rate their symptoms and early warning signs over time. In the Symptom Tracker, users are presented with 17 different symptoms and behaviours and are asked to respond via a “Yes/No” format as to whether they have recently experienced these. Users who respond with a

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.



“Yes” are then presented with a 10-point scale (4-point scale for the early warning sign tracker) to rate the severity or frequency of the associated symptoms, with advice on how to manage these symptoms displayed. Psychoeducation on mental health, medication and mental health services is provided in an ‘Information' section. To encourage adherence with medication, users are encouraged to log and track their medication in the ‘Pill Tracker' section. Users are able to set daily alerts to remind them to log whether they have taken their medication. My Journey 3 also features weekly discrete notifications to encourage engagement with the app, which can be disabled at the users' preference. The key components of My Journey 3 are summarised in table 1, with further details available in the protocol paper.32

Prior to the feasibility trial reported in this paper, My Journey 3 was tested by EIP service users in laboratorybased usability tests and in a 1-month field study. The final content of My Journey 3 was then refined based on feedback from individual interviews with the participating EIP service users and clinicians. No changes were made to the content of My Journey 3 during the feasibility RCT. A major technical update to My Journey 3 was carried out in January 2018 to fix compatibility issues with older versions of Android operating systems. This did not require any changes to the trial design.

Delivery

Following assignment to the treatment group, participants engaged in individual training sessions with a trial researcher and a supporting EIP service clinician. Training sessions were intended to take place within 6weeks of the participants' initial baseline assessment, and lasted for approximately 2 hours. During these sessions, the researcher downloaded My Journey 3 onto the participants' Smartphone and gave a demonstration of the app and its main functions. Participants were then encouraged to input appropriate information to specific sections of My Journey 3 with the help of the supporting EIP service clinician in attendance. Following this session, it was hoped that all participants had initial personal recovery plans, relapse prevention plans and crisis plans stored on My Journey 3.

Table 1 Key sections of the My Journey 3 Smartphone app

Section

Features

Purpose

My recovery plan

Things I can do to keep well My goals

To encourage users to have regular routines, track activities, set reminders and plan how to achieve long-term goals

My relapse prevention plan

Coping with triggers

Coping with early warning signs Coping with a crisis

Crisis contacts

To help users identify, monitor and cope with triggers and early warning signs To help users create a ‘relapse plan' to follow in times of crisis

How are you doing?

My mood

My early warning signs

My tracker

For users to monitor symptoms, behaviours and early warning signs and track these experiences over time

Pill tracker

To log whether users have taken their medication each day

Information

Medication information Useful websites Emergency services Jargon buster

To provide users with useful information and external links on medication and mental health

To identify local emergency services in a time of crisis

To provide a glossary of terms that are commonly used in mental health care


BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Participants had access to My Journey 3 on their own Smartphone from the training session until the 12-month time point. Researchers recommended that participants used My Journey 3 at least once a week, but participants had a free choice in how and when they used My Journey

To encourage user engagement with My Journey 3 during the trial, supporting EIP service clinicians were asked to provide regular support and encouragement to service users who had access to My Journey 3. Clinicians were asked to discuss recovery goals and relapse prevention plans in routine appointments with participants, and assist with entering these into the appropriate My Journey 3 sections. Clinicians had an existing understanding of self-management approaches from their clinical training and practice, and would be able to provide appropriate advice with the intervention components of My Journey 3, but they received no formal training on how to implement My Journey 3 into their clinical work. Clinicians’ understanding of operating My Journey 3 was from the training sessions only. Clinician support for My Journey 3 as part of the trial was not manualised or incentivised.

Participants were encouraged to contact the trial researcher in the case of technical problems with My Journey 3. The researcher contacted participants a week after the training session to check that My Journey 3 had been functioning without issues and invited any questions about the app. No further prompts were instigated by the researcher during the trial.

Treatment as usual

All participants received TAU regardless of group allocation. TAU for a person under the care of EIP services typically involves regular meetings with a care coordinator, access to a psychiatrist, psychiatric medication and a range of psychological interventions. EIP services are encouraged to deliver self-management programmes that include advice on symptom management, crisis planning and relapse prevention, generally delivered with paper-and-pen tools if at all.34 None of the participating EIP services offered digital interventions or Smartphone apps as part of routine care during the study period, and structured self-management support, including the relapse prevention work recommended in EIP contexts, was inconsistently implemented.

Patient and participant involvement

The development of My Journey 3 has been guided by the input of people with lived experience of psychosis. Initial development of the design and content involved a collaboration between researchers, experts in digital health and service users. Service users provided further input into the design and functionality of MyJourney 3 by providing feedback after taking part in laboratory-based tests and a field study.

Outcomes

Participant data were collected from numerous sources including participant assessments, patient records and anonymous My Journey 3 usage reports. There were no pre-specified criteria for assessing trial feasibility and intervention acceptability.

Questionnaire measures

Proposed outcome measures for a future trial were assessed at structured face-to-face assessments with a trained researcher at three time points: baseline, 4 months post baseline and 12 months post baseline. At all meetings, participants completed self-report questionnaires that have been previously used with people with first-episode psychosis. Participants were given £20 as a thank you for completing the assessment at each time point.

At each assessment, we collected sociodemographic data including age, gender, ethnicity, accommodation and living situation, employment status, educational attainment, Smartphone use and use of other mental health apps. The following self-report measures were also collected: social outcomes (Social Outcomes Index (SIX),38 score 0-6: higher score=better social outcomes), self-efficacy (Mental Health Confidence Scale (MHCS),39 score 16-96: higher score=greater empowerment), selfrated recovery (Questionnaire about the Process of Recovery (QPR),40 intrapersonal score 0-68, interpersonal score 0-20: higher score=greater recovery), mental well-being (Warwick-Edinburgh Mental Well-Being Scale (WEMWBS),41 score 14-70: higher score=greater wellbeing) and quality of life and satisfaction with treatment (DIALOG scale,42 score 1-7: higher score=greater quality of life/satisfaction with treatment).

Clinical structured interviews were also conducted with each participant by the researcher, to assess psychopathology, using the Positive and Negative Syndrome Scale (PANSS).43 Higher PANSS scores are indicative of greater severity of each symptom domain.

Participants’ engagement with EIP services were measured using the Service Engagement Scale (SES),44 completed by EIP service clinicians known to each participant, typically care co-coordinators. Clinicians completed the SES at baseline and 12 months later, regardless of whether participants attended the 12-month assessment. Higher SES scores are indicative of poorer user engagement with EIP services.

Patient records

Clinical data were extracted from patient records at baseline and at the 12-month time point. Clinical measures included most recent diagnosis and use of EIP services, other community mental health teams and acute mental health services in the previous 12 months.

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


The proposed primary outcome for a future RCT (relapse of psychosis) was operationalised as an admission to an acute mental health service (inpatient psychiatric ward, crisis house, crisis resolution team or acute day care service) during the 12-month trial period as indicated in patient records. This definition of relapse has been used previously in a recent trial of a self-management intervention.45

My Journey 3 use

To assess acceptability of the intervention and user engagement, My Journey 3 usage data were collected for all participants in the treatment group from the training session until the 12-month time point. Whenever users had Wi-Fi internet access on their Smartphone, My Journey 3 automatically uploaded encrypted usage data to a secure server. Data collected included a record of each time the user opened My Journey 3, whether this was in response to a prompt and which components they used. To ensure confidentiality, personal or identifiable data such as text or responses to each sections were not collected.

Acceptability

Feedback was obtained through semi-structured interviews as part of a nested qualitative study. Individual interviews were conducted at the 4-month time point with both service user participants that received My Journey 3 and supporting clinical staff.

Analysis

Participant demographic and clinical characteristics, My Journey 3 usage, and rates of participant recruitment and retention were summarised using descriptive statistics. As this was a feasibility RCT, it was not powered to assess the effectiveness of the intervention. Statistical analyses of participant outcome measures were conducted to pilot the methods of analysis for a fully powered effectiveness trial. Logistic regression was used to explore the impact of the My Journey 3 intervention on relapse. Linear regression was used to examine the potential effect of the intervention on continuous outcome measures at 4 months and 12 months separately. We report the effect estimates and corresponding 95% CIs only for unadjusted analyses and for analyses adjusting for the baseline measure of the outcome in question. All analyses were performed using STATA V.14 after completion of the final participant assessment. No interim analyses were conducted.

Qualitative data were coded to themes based on the Acceptability of Healthcare Interventions framework.46 Results of the nested qualitative study exploring the acceptability of My Journey 3 and drivers of engagement and non-adherence will be reported in full elsewhere. Here, we provide a short summary of findings.

RESULTS

Feasibility of trial design

Participant flow is detailed in the CONSORT diagram (figure 2). A total of 40 participants was recruited and randomised (20 to My Journey 3, 20 to TAU) over a 7-month period from March 2017 to September 2017. Participants were recruited until the required number of 40 was obtained: we do not therefore have a full assessment of the proportion of the teams’ caseload who could have been recruited to a full trial, nor do we know the proportion of approached EIP services users that did not meet eligibility criteria or declined involvement in the trial.

Among those recruited to the trial, attrition rates were generally low: 83% (33/40) and 75% (30/40) of participants successfully attended and completed follow-ups at 4 months and 12 months, respectively. At both time points, the follow-up rate was lower in the control group (4months: 65% compared with 100%, 12months: 70% compared with 80%). Patient record data were available for all participants at baseline and for 95% of the sample (38/40) at the 12-month time point. Completion rates of the SES by clinicians were higher at baseline (90%) than at the 12-month time point (67.5%). Follow-up assessments were conducted from July 2017 to October 2018.

All participants in the treatment group attended a training session with a researcher and had access to My Journey 3 during the trial. Issues with Smartphone compatibility initially prevented three participants from downloading My Journey 3. Following an update to the system, two of the participants were able to install and access My Journey 3 on their own Smartphones. Two participants were provided with Smartphones with My Journey 3 pre-installed (the app was still incompatible on one participant’s Smartphone despite the update; another participant no longer owned an Android Smartphone after entering the trial). The median length of time from trial enrolment to having access to My Journey 3 was 14 weeks (IQR 11 to 17), longer than the planned time of 6 weeks. Participants had access to My Journey 3 for a median of 38.1 weeks (IQR 34.8 to 40.7). There were no reported privacy breaches.

My Journey 3 usage data were collected for all participants following the training session, with 500 different data entries available for analysis. Within the 500 data entries, 27 (5.4%) were corrupt and were subsequently removed from the analysis. The unusable data can grouped into two types. The first, duplicates of previous data entries that were subsequently removed. The second, entries where the times were implausible (eg, the end time of using My Journey 3 was recorded as occurring before the start time). In addition, a further issue caused errors with accurately recording My Journey 3 usage data of ‘My Recovery Plan’ and ‘My Relapse Plan’ sections. As a result, we were unable to accurately conclude how often participants used these sections.

One participant randomised to the control group was wrongly given access to My Journey 3. For the purpose

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.



of the statistical analysis, they are classed as a control participant.

Sample characteristics

A summary of demographic and clinical characteristics of the sample is displayed in table 2. The sample was predominantly male (n=28, 70%). The mean age of the sample was 29.7 years (SD 9.78) and similar to that of UK cohorts of EIP service users at first presentation.47 48 Six participants were over the age of 35, with these participants spread evenly across the two groups. Most participants had a diagnosis of a schizophrenia, schizotypal or delusional disorder (ICD code F20-F29) and were not in paid employment. A quarter of the sample (n=10, 25%) had completed a university degree. Eight (20%) participants had previously used a mental health app.

My Journey 3 use

The level of My Journey 3 use was highly skewed. The median number of times My Journey 3 was used per participant during the trial was 16.5 (IQR 8.5 to 23). Participants accessed My Journey 3 on a median of 3.22% (IQR 1.89 to 6.36) of the days it was available to them, equating to My Journey 3 being used on average once every 31 days (IQR 15.7 to 52.9). Participants spent a median of 26.8 min (IQR 18.3 to 57.3) in total using My Journey 3 over the course of the trial. Eight participants (40%) used My Journey 3 for longer than 30 min in total.

Five participants (25%) were still using MyJourney 3 six months after downloading it; however, one participant never used the app after the training session (figure 3). Half of the participants (n=10) stopped using MyJourney 3 within the first 3 months after the training session.

The average number of uses by participants for each My Journey 3 component is displayed in table 3. The most frequently accessed section was the “How are you doing?” Symptom Tracker section (median uses 3; IQR 1 to 6); however, data on how frequently users accessed ‘My Recovery Plan’ and ‘My Relapse Plan’ are unavailable. The ‘Information’ section was accessed the fewest times, with 25% (n=5) of participants in the treatment group never using that section following the training session. Just over 7% of MyJourney 3 uses were initiated following a prompt from the app.

My Journey 3 acceptability

Qualitative interviews were conducted with all participants who received MyJourney 3 and the majority of clinical staff who supported its delivery. In general, most service user participants found My Journey 3 to be acceptable, and a number of participants reported a clear benefit from using it. Barriers affecting use were identified including

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http:ZZbmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Table 2 Key demographic and clinical characteristics of the sample at baseline

Control (n=20)

My Journey 3 (n=20)

Age (years)-mean (SD), (min, max)

30 (10.1), (18.8, 64.7)

29.4 (9.7), (17.6, 52.4)

Gender

Female

7 (35%)

5 (25%)

Ethnicity

White British

6 (30%)

8 (40%)

Any other white/Mixed white

2 (10%)

1 (5%)

Black African

5 (25%)

3 (15%)

Black Caribbean

1 (5%)

1 (5%)

Black Other

1 (5%)

0

Asian Indian

1 (5%)

0

Asian Other

1 (5%)

2 (10%)

Other/Mixed other

3 (15%)

3 (15%)

Education

Undergraduate degree

6 (30%)

4 (20%)

Some University but no degree

3 (15%)

2 (10%)

Higher National Degree or professional qualification

2 (10%)

1 (5%)

A Levels or equivalent

3 (15%)

4 (20%)

GCSEs or equivalent

4 (20%)

6 (30%)

No qualifications

1 (5%)

3 (15%)

Missing

1 (5%)

0

Employment status

Employed — more than 16 hours a week

4 (20%)

4 (20%)

Employed — less than 16 hours a week

0

2 (10%)

Voluntary work

3 (15%)

3 (15%)

In study or training

1 (5%)

1 (5%)

Unemployed or exempt due to disability

8 (40%)

8 (40%)

Missing

4 (20%)

2 (10%)

Primary diagnosis (ICD-10 code)

F10—F19: Mental and behavioural disorder due to psychoactive substance use

1 (5%)

0

F20-F29: Schizophrenia, schizotypal and delusional disorder

16 (80%)

13 (65%)

F30-F39: Mood disorder

1 (5%)

5 (25%)

Missing

2 (10%)

2 (10%)

Admission to an acute mental health service in previous year

Yes

11 (55%)

10 (50%)

SIX—mean (SD), (min, max)

3.2 (1.5), (0, 6)

3.6 (1.5), (1, 6)

MHCS—mean (SD), (min, max)

59.7 (17.8), (16, 82)

61.2 (12.6), (38, 78)

QPR—mean (SD), (min, max)

Intrapersonal

45.7 (12), (22, 68)

42.2 (10.6), (24, 60)

Interpersonal

13.7 (2.7), (9, 19)

12.9 (3.4), (5, 19)

WEMWBS—mean (SD), (min, max)

43.4 (11.6), (25, 69)

40.3 (10.2), (23, 57)

DIALOG—mean (SD), (min, max)

Quality of life

4.5 (1), (2.8, 6.5)

4.4 (0.8), (3, 5.7)

Treatment satisfaction

5.4 (0.7), (4.3, 7)

4.8 (0.7), (3.7, 6)

Continued


BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Table 2 Continued

Control (n=20)

My Journey 3 (n=20)

PANSS—mean (SD), (min, max)

Positive

10.9 (5), (7, 22)

11.3 (4.2), (7, 19)

Negative

10.7 (2.5), (7, 19)

11.8 (4.5), (7, 20)

General

26.6 (6), (17, 39)

26.2 (8), (16, 46)

SES—mean (SD), (min, max)

11.3 (7.9), (0, 26)

9.6 (7), (0, 23)

All statistics are reported N (%) unless otherwise specified. group participants, one treatment group participant.

Missing data: PANSS scores—one control group participant, SES—three control


a lack of clinician support and concerns around data privacy. A key theme for staff was that they often did not have the time to provide regular support to participants with My Journey 3.

Participant outcomes

No research-related serious adverse events were recorded. Psychotic and general symptoms (measured by the PANSS) were generally low at all times for both groups, suggesting a stable sample. Summary statistics and estimated effect sizes of participant outcomes are displayed in table 4. Inspection of the effect sizes and confidence intervals suggest that were no obvious differences for any outcome measure between the treatment and control group at either time point.

Of the 38 participants whose patient records data were available, only five experienced a relapse during the trial, as indicated by using an acute mental health service. In the treatment group, 15% of participants (3/20) experienced

a relapse during the trial period compared with 11% (2/18) in the control group. We found no evidence of a difference in relapse between the two groups (OR 1.41; 95% CI 0.21 to 9.58), but did not have sufficient power for an informative test.

DISCUSSION

The present study examined the feasibility of conducting an RCT of a supported self-management Smartphone app in EIP services. My Journey 3 aims to facilitate recovery and prevent relapse primarily via the digital delivery of previously developed paper-and-pen self-management tools. The trial indicates that recruitment and retention in an RCT evaluating My Journey 3 is feasible, and that MyJourney 3 can be delivered in EIP services. The level of My Journey 3 use was relatively low across the trial period.

Building on from extensive preliminary work with NHS staff and service users, adults with lived experience of psychosis and experts in digital health, we were able to successfully develop a self-management Smartphone app that can be used in EIP services. My Journey 3 appeared to be safe with no related serious adverse events reported. My Journey 3 was successfully delivered to all participants in the treatment group; however, technical problems with the intervention caused significant delays in providing access. Prior to any future evaluations, technical problems with My Journey 3 will need to be identified and fixed to ensure the intervention is implemented as intended.

My Journey 3 use varied considerably between participants, with only a small proportion of participants frequently engaging with the app after obtaining access to it. This raises questions about whether use was at a level where it is likely that useful self-management activities were taking place: certainly not enough time was spent regularly enough for participants to be engaging in detailed monitoring of symptoms and early warning signs, tracking medication and activities and referring to crisis or recovery plans. Despite that, 40% of participants used My Journey 3 for a minimum of 30 minutes which could be an adequate amount of time for users to effectively monitor relapse signs and follow a crisis plan when needed. We have not found evidence on how regularly EIP service users make use of pen-and-paper self-management

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Table 3 Participant use of My Journey 3 and various sections

Number of times used per participant

Days used while having access to My Journey 3 (%)

Participants that did not use app or section—n (%)

My Journey 3

16.5 (8.5 to 23)

3.22 (1.89 to 6.36)

1 (5%)

How are you doing?

3 (1 to 6)

1.08 (0.4 to 2.12)

3 (15%)

Pill tracker

2 (1 to 3.5)

0.73 (0.36 to 1.07)

3 (15%)

Information

1 (0 to 2.5)

0.48 (0.18 to 0.7)

5 (25%)

All median (IQR), except when stated.


interventions delivered in routine settings, and this was not measured in our trial. Long-term engagement with My Journey 3 appears a challenge, but low levels of app use is a common phenomenon with market research showing that 62% of users stop using Smartphone apps after 10 or fewer uses.49

Age has been shown to be an important factor linked to engagement with mental health apps and general Smartphone use,50 and could partially explain differences in user engagement of My Journey 3. The treatment group, however, featured only a small number of participants from older age groups. We therefore lack informative data regarding app engagement for older participants and we are accordingly unable to explore if engagement and pattern of use of My Journey 3 varied between age groups.

Participant retention for research data collection was high, with 75% of the sample attending the 12-month follow-up assessment, and is comparable with other Smartphone app studies.51 Completion rates of the SES by EIP service clinicians were much lower at the 12-month follow-up in comparison with baseline, potentially due to staff changes and participants being discharged from services. Recruitment strategies were largely successful; however, data are lacking on overall proportion of caseload recruited, reasons for non-inclusion and the numbers that were assessed for eligibility, thus limiting the conclusions we can make regarding trial feasibility.

The trial was not powered to detect effectiveness, and, as expected with our small number of participants, we found no significant differences between groups on any outcomes, with CIs generally including substantial effects in either direction. Accordingly, we cannot draw any conclusions regarding the potential impact of MyJourney 3 as a mental health intervention. The proposed primary outcome for a full-scale trial, relapse as defined by use of an acute mental health service during the trial period, was marked by low event rates. Only five participants (12.5%) experienced a relapse during the 1-year follow-up period, compared with expected levels of 12% to 47%.52 Consideration should be given to whether relapse, or our measure of relapse, is an appropriate outcome for a future RCT of this intervention. Symptom severity or alternatively patient-valued outcomes of personal recovery that selfmanagement interventions have been shown to benefit may be more suitable primary outcomes in a future large-scale trial.12

Strengths and limitations

My Journey 3 has been developed with extensive stakeholder input, and the intervention has been tested through laboratory testing and a field study prior to the feasibility RCT. In comparison with previous studies,51 participants had access to the app for a longer period of time. Participants’ app use and usage data may be more reflective of real-world use as a result. Participant data were also collected from a wide range of methods including from participant assessments and patient records. The proposed primary outcome for a future RCT (relapse) was measured objectively and data were obtained for 95% of participants.

We recruited until the required number of participants was obtained rather than screening caseloads objectively: as a result, we are not aware of the proportion eligible who were recruited, reasons for non-eligibility and how many EIP service users declined to take part and why. This limits our understanding of how feasible conducting a large-scale trial of this intervention would be. In addition, there were problems with the usage data, which impacts the reliability of our conclusions regarding how often participants engaged with My Journey 3.

The trial did not feature an active digital placebo for the control group, meaning that non-specifics of Smartphone use could not be controlled for. Furthermore, data were not collected during the study period from either group regarding frequency of completing recovery work such as relapse prevention plans, recovery plans or crisis plans either in paper-and-pen or digital format, limiting our understanding of whether access to My Journey 3 facilitated increased access to self-management activities.

Although clinicians were encouraged to support participants with My Journey 3, support was not manualised and clinicians did not have personal access to the app or associated data, potentially limiting the level and quality of the support offered and therefore user engagement. Future developments of My Journey 3 should focus on effective implementation and delivery within healthcare settings, and there should be considerations on how to facilitate secure data-sharing between My Journey 3 and healthcare records or other secure web-based platforms dependent on user consent, which is likely to increase clinician engagement with the app and its utility.53

We did also not define pre-specified criteria for assessing the feasibility of a RCT and the acceptability of My Journey 3. Instead, we will consider all findings from

BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Table 4 Summary statistics and unadjusted and adjusted treatment effects

4-month scores

Control (n=13)

My Journey 3 (n=20)

Unadjusted analysis

Analysis adjusted for baseline score

Mean (SD)

Mean (SD)

Estimated difference

95% CI

Estimated difference

95% CI

SIX (Social Outcomes)

3.3 (1.9)

3.6 (1.3)

0.29

-0.84 to 1.43

0.16

-0.6 to 0.92

MHCS

(Mental Health Confidence)

66.4 (12.7)

63 (15.8)

-3.43

-14.1 to 7.25

-4.81

-14.88 to 5.25

QPR (Recovery)

Intrapersonal

47.8 (10.6)

43.2 (12.2)

-4.57

-13 to 3.87

-2.01

-8.43 to 4.49

Interpersonal

13.9 (2.4)

13.2 (2.3)

-0.72

-2.39 to 0.95

-0.42

-1.97 to 1.13

MHCS

(Mental Health Confidence)

46.1 (9.9)

44 (11.3)

-2.08

-9.9 to 5.74

-0.19

-7.28 to 6.9

DIALOG

Quality of life

4.4 (1.2)

4.5 (0.6)

0.07

-0.58 to 0.71

0.18

-0.38 to 0.74

Treatment satisfaction

5.4 (0.7)

5 (0.5)

-0.38

-0.83 to 0.06

-0.17

-0.6 to 0.25

PANSS (Symptom

Severity)

Positive

9.3 (2.9)

11.4 (5.1)

2.09

-1.24 to 5.4

1.9

-0.49 to 4.3

Negative

10 (2.3)

11.1 (3.9)

1.05

-1.51 to 3.62

0.54

-1.6 to 2.67

General

23 (4)

24 (6.7)

1.21

-3.19 to 5.61

1.35

-2.68 to 5.37

Control (n=14)

My Journey 3 (n=16)

Unadjusted analysis

Analysis adjusted for baseline score

12-month scores

Mean (SD)

Mean (SD)

Estimated difference

95% CI

Estimated difference

95% CI

SIX (Social Outcomes)

3.2 (1.9)

3.5 (1.5)

0.29

-0.97 to 1.54

0.29

-0.73 to 1.3

MHCS

(Mental Health Confidence)

66.2 (14.1)

71.1 (12.1)

4.81

-5 to 14.62

3.03

-6.04 to 12.1

QPR (Recovery)

Intrapersonal

47.3 (11.5)

49.5 (11.1)

2.2

-6.25 to 10.7

3.21

-4.12 to 10.5

Interpersonal

13.6 (3.4)

15.1 (3.3)

1.44

-1.09 to 3.96

1.62

-0.89 to 4.12

MHCS

(Mental Health Confidence)

45.6 (11.3)

49.3 (9.7)

3.61

-4.24 to 11.46

5.03

-1.67 to 11.7

DIALOG

Quality of life

4.7 (0.9)

5 (0.7)

0.28

-0.31 to 0.87

0.24

-0.33 to 0.81

Treatment satisfaction

5.3 (1)

5.2 (1.2)

-0.12

-0.93 to 0.69

0.31

-0.42 to 1.04

PANSS (Symptom

Severity)

Positive

9.5 (2.1)

10.2 (2.1)

0.69

-0.98 to 2.36

0.88

-0.62 to 2.38

Negative

10.2 (2.2)

10.9 (3.3)

0.77

-1.51 to 3.05

0.14

-1.56 to 1.84

General

23.5 (5.4)

22.1 (3.5)

-1.38

-4.82 to 2.07

-1

-4.57 to 2.55

SES (Engagement with Services)

10 (6.2)

9.5 (8)

-0.4

-6.08 to 5.28

3.11

-1.57 to 7.79

Continued


BMJ Open: first published as 10.1136/bmjopen-2019-034927 on 26 August 2020. Downloaded from http://bmjopen.bmj.com/ on April 3, 2021 by guest. Protected by copyright.


Table 4 Continued

Control        My Journey 3                                   Analysis adjusted for baseline

(n=14)         (n=16)         Unadjusted analysis              score

12-month scores                              Estimated                      Estimated

Mean (SD) Mean (SD) difference 95% CI           difference 95% CI

Estimated differences and associated 95% Confidence Intervals from linear regression models with thecontrol group as reference. Missing data: 4-month PANSS scores - one control group participant, one treatment group participant. 12-month PANSS scores - two control group participants. Note: 12-month SES data available for 13 control group participants, and 14 treatment group participants.


the trial, app usage data and feedback from qualitative interviews yet to be reported in determining whether My Journey 3 will be evaluated in a full-scale trial. This allows all data from the RCT to be thoroughly considered, but may be a less objective approach in determining feasibility than using pre-defined criteria. Although the trial was not designed to assess intervention effectiveness, participants and trial researchers were not blinded to group allocation, and as such could have led to an inflation of any observed effects.

Finally, the sample consisted of Android Smartphone users who were generally stable and in an appropriate stage of recovery to consider using a self-management Smartphone app. Participants may therefore not be representative of all EIP service users. Furthermore, contact with a researcher within a trial context could have led to increased intervention engagement that would not occur in a real-world clinical environment.

CONCLUSIONS

We developed and delivered a self-management Smartphone app for first-episode psychosis in a trial context. Participants were successfully recruited, most engaged at least to some extent with the intervention, and they had high follow-up rates over the 1-year trial period. Based on the data presented, the trial methods appear feasible. My Journey 3 was shown to be safe, but the level of use was lower than anticipated thus potentially limiting its utility.

If My Journey 3 is to be further tested in a research setting, attention needs to be given to engagement, a challenge associated with many digital tools in mental health.54 Further usability testing in laboratory and field settings may be a means to improving engagement. Other potential strategies include making more efforts to engage clinicians as well as service users with My Journey 3 by giving them access to the tool and to aspects of the planning and monitoring that service users conduct through it. The app could also potentially be offered as part of a blended approach to self-management, with pen-and-paper tools also used and as a whole service strategy for implementation of selfmanagement. Refinements required before participating to a full trial including participant and assessor blinding and manualised clinician support should be considered prior to conducting a future RCT.

Twitter Thomas Steare @tomsteare, Puffin O'Hanlon @PuffinOH, Michelle Eskinazi @MichEskinazi, David Osborn @osborn_ucl and Sonia Johnson @soniajohnson

Acknowledgements The ARIES research team are grateful to their software collaborators MyOxygen for their technical development and hosting of My Journey 3 and to Ali Mousa for his valuable contribution to the development of the original My Journey app. We are grateful to Max Birchwood for his permission to incorporate ‘Back in the Saddle' into My Journey 3. We are grateful to Rachel Perkins for her permission to adapt the Personal Recovery Plan resource and incorporate in to My Journey 3.

Contributors SJ is the chief investigator, based at University College London, DO the co-chief investigator and TS the project manager. The trial design was developed by SJ, DO, BL-E and PO. SA, HR, PO and ME have led on the development of the intervention. TS conducted the statistical analysis, with advice from RJ. TS wrote the draft of the paper, which was revised and approved by all authors. All authors approved the final manuscript.

Funding The research is funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust (NIHR CLAHRC North Thames). SJ, DO and BL-E are supported by the NIHR Mental Health Research Policy Unit, the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames and the UCLH Biomedical Research Centre.

Disclaimer The views expressed in this article are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing interests None declared.

Patient consent for publication Not required.

Ethics approval National Research Ethics Service Committee London—Brent (Research Ethics Committee reference: 15/LO/1453).

Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement No data are available. The datasets generated during and/or analysed during the current study will be available 2 years after the trial end.

Open access This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/ licenses/by/4.0/.

ORCID iDs

Thomas Steare http://orcid.org/0000-0002-3881-2018 Sonia Johnson http://orcid.org/0000-0002-2219-1384

REFERENCES

Leucht S, Heres S. Epidemiology, clinical consequences, and psychosocial treatment of nonadherence in schizophrenia. J Clin Psychiatry 2006;67 Suppl 5:3-8.

Haddock G, Eisner E, Boone C, et al. An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia. J Ment Health 2014;23:162-5.

Mueser KT, Meyer PS, Penn DL, et al. The illness management and recovery program: rationale, development, and preliminary findings. Schizophr Bull 2006;32 Suppl 1:S32-43.

Mueser KT, Corrigan PW, Hilton DW, et al. Illness management and recovery: a review of the research. PS 2002;53:1272-84.

Cook JA, Copeland ME, Jonikas JA, et al. Results of a randomized controlled trial of mental illness self-management using wellness recovery action planning. Schizophr Bull 2012;38:881-91.

Fardig R, Lewander T, Melin L, et al. A randomized controlled trial of the illness management and recovery program for persons with schizophrenia. Psychiatr Serv 2011;62:606-12.

Lean M, Fornells-Ambrojo M, Milton A, et al. Self-management interventions for people with severe mental illness: systematic review and meta-analysis. Br J Psychiatry 2019;214:1-9.

NICE Guidance. Psychosis and schizophrenia in adults: prevention and management. Available: https://www.nice.org.uk/guidance/ cg178/ifp/chapter/peer-support-and-self-management [Accessed 30 Jan 2020].

Whitehead L, Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. J Med Internet Res 2016;18:e97.

Poushter J. Smartphone ownership and Internet usage continues to climb in emerging economies. URL, 2016. Available: http://www. pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ [Accessed 30 Jan 2019].

Wigginton C, Curran M, Broedeur C. Global mobile consumer trends. 2nd edition, 2017. https://www2.deloitte.com/content/dam/Deloitte/ us/Documents/technology-media-telecommunications/us-global-mobile-consumer-survey-second-edition.pdf

Firth J, Cotter J, Torous J, et al. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: a meta-analysis of cross-sectional studies. Schizophrenia Bull 2015;42:448-55.

Robotham D, Satkunanathan S, Doughty L, et al. Do we still have a digital divide in mental health? A five-year survey follow-up. J Med Internet Res 2016;18:e309.

Naslund JA, Aschbrenner KA, Bartels SJ. How people with serious mental illness use smartphones, mobile apps, and social media. Psychiatr Rehabil J 2016;39:364-7.

Aref-Adib G, O'Hanlon P, Fullarton K, et al. A qualitative study of online mental health information seeking behaviour by those with psychosis. BMC Psychiatry 2016;16:232.

Berry N, Lobban F, Bucci S. A qualitative exploration of service user views about using digital health interventions for self-management in severe mental health problems. BMC Psychiatry 2019;19:35. Proudfoot J. The future is in our hands: the role of mobile phones in the prevention and management of mental disorders. Aust N Z J Psychiatry 2013;47:111-3.

Gottlieb JD, Romeo KH, Penn DL, et al. Web-based cognitive-behavioral therapy for auditory hallucinations in persons with psychosis: a pilot study. Schizophr Res 2013;145:82-7.

Granholm E, Ben-Zeev D, Link PC, et al. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull 2012;38:414-25.

Ben-Zeev D, Kaiser SM, Brenner CJ, et al. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013;36:289-96.

Alvarez-Jimenez M, Bendall S, Lederman R, et al. On the HORYZON: moderated online social therapy for long-term recovery in first episode psychosis. Schizophr Res 2013;143:143-9.

Bonet L, Izquierdo C, Escarti MJ, etal. Use of mobile technologies in patients with psychosis: a systematic review. Rev Psiquiatr Salud Ment 2017;10:168-78.

Rus-Calafell M, Schneider S. Are we there yet?!—a literature review of recent digital technology advances for the treatment of early psychosis. mHealth 2020;6:3.

Camacho E, Levin L, Torous J. Smartphone apps to support coordinated specialty care for prodromal and early course schizophrenia disorders: systematic review. J Med Internet Res 2019;21:e16393.

ORIGINAL RESEARCH

published: 23 December 2016 doi: 10.3389/fpsyt.2016.00196


OPEN ACCESS

Edited by:

Jerome Favrod,

University of Applied Sciences and Arts of Western Switzerland, Switzerland

Reviewed by: Karsten Heekeren,

University of Zurich, Switzerland Kim T Mueser, Boston University, USA

Correspondence:

Neil Thomas neilthomas@swin.edu.au

Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 19 September 2016 Accepted: 25 November 2016 Published: 23 December 2016

Citation:

Thomas N, Farhall J, Foley F, Leitan ND, Villagonzalo K-A, Ladd E, Nunan C, Farnan S, Frankish R, Smark T, Rossell SL, Sterling L, Murray G, Castle DJ and Kyrios M (2016) Promoting Personal Recovery in People with Persisting Psychotic Disorders: Development and Pilot Study of a Novel Digital Intervention.

Front. Psychiatry 7:196. doi: 10.3389/fpsyt.2016.00196


Promoting Personal Recovery in People with Persisting Psychotic Disorders: Development and Pilot Study of a Novel Digital Intervention

Neil Thomas12*, John Farhall3'4, Fiona Foley1, Nuwan Dominic Leitan1, Kristi-Ann Villagonzalo1, Emma Ladd5, Cassy Nunan5, Sue Farnan5, Rosalie Frankish5, Tara Smark5, Susan L. Rossell1,2,6, Leon Sterling7,8, Greg Murray1, David Jonathon Castle6,9 and Michael Kyrios1,10

1 Centre for Mental Health, Swinburne University of Technology, Hawthorn, VIC, Australia, 2 Monash Alfred Psychiatry Research Centre, Monash University and The Alfred, Melbourne, VIC, Australia, 3 Department of Psychology and Counselling, La Trobe University, Melbourne, VIC, Australia, 4 Northwestern Mental Health, Royal Melbourne Hospital, Melbourne, VIC, Australia,5 Wellways Australia, Melbourne, VIC, Australia, 6Department of Psychiatry, St Vincent’s Hospital, Fitzroy, VIC, Australia, 7 Centre for Design Innovation, Swinburne University of Technology, Hawthorn, VIC, Australia, 8Department of Computing and Information Systems, University of Melbourne, Parkville, VIC, Australia, 9 Department of Psychiatry, University of Melbourne, Parkville, VIC, Australia, 10 Research School of Psychology Australian National University, Canberra, ACT, Australia

Background: For people with persisting psychotic disorders, personal recovery has become an important target of mental health services worldwide. Strongly influenced by mental health service consumer perspectives, personal recovery refers to being able to live a satisfying and contributing life irrespective of ongoing symptoms and disability. Contact with peers with shared lived experience is often cited as facilitative of recovery. We aimed to develop and pilot a novel recovery-based digitally supported intervention for people with a psychotic illness.

Methods: We developed a website to be used on a tablet computer by mental health workers to structure therapeutic discussions about personal recovery. Central to the site was a series of video interviews of people with lived experience of psychosis discussing how they had navigated issues within their own recovery based on the ConnectednessHope-Identity-Meaning-Empowerment model of recovery. We examined the feasibility and acceptability of an 8-session low intensity intervention using this site in 10 participants with persisting psychotic disorders and conducted a proof-of-concept analysis of outcomes.

Results: All 10 participants completed the full course of sessions, and it was possible to integrate use of the website into nearly all sessions. Participant feedback confirmed that use of the website was a feasible and acceptable way of working. All participants stated that they would recommend the intervention to others. Post-intervention, personal recovery measured by the Questionnaire for the Process of Recovery had improved by an average standardized effect of d = 0.46, 95% CI [0.07, 0.84], and 8 of the 10 participants reported that their mental health had improved since taking part in the intervention.

Conclusion: In-session use of digital resources featuring peer accounts of recovery is feasible and acceptable and shows promising outcomes. A randomized controlled trial is the next step in evaluating the efficacy of this low intensity intervention when delivered in conjunction with routine mental health care.

Keywords: schizophrenia, psychosis, personal recovery, mental health services, low intensity interventions, digital health, tablet computers, peer support

INTRODUCTION

Psychotic disorders represent one of the leading causes of disability and need for ongoing health care in working age adults. In Australia, for example, approximately 4.5 per 1,000 people receive specialist mental health care for a psychotic disorder each year, not including those treated exclusively by private psychiatrists or in primary care (1). In spite of the routine use of antipsychotic medication, and efforts over the past two decades to ensure psychosis is promptly treated at its first emergence, health outcomes remain unsatisfactory for many. For example, the 2010 Australian National Survey of High Impact Psychosis reported that 92% of people seen in specialist mental health services have either recurring or unremitting episodes, 62% experience continuous symptoms, and 90% have deteriorated social functioning (1).

For individuals who experience persisting symptoms and disability, personal recovery has become an important target of mental health services internationally (2-7). Often contrasted with the traditional treatment targets of minimizing symptoms (clinical recovery) or improving social and occupational functioning (functional recovery), the concept of personal recovery has developed from the perspectives of people who use mental health services to prioritize more personal and subjectively meaningful goals of treatment (8-10). A widely used definition is that recovery is “a deeply personal, unique process of changing one's attitudes, values, feelings, goals, skills and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with limitations caused by the illness” (11). The literature on personal recovery has primarily been based on mental health service consumer narratives about the processes that have been most relevant in their own recovery. Although typically characterized as an individual journey, there has been convergence on identification of processes involved in recovery. An influential synthesis of qualitative studies on recovery has highlighted five themes, summarized by the acronym Connectedness-HopeIdentity-Meaning-Empowerment (CHIME): (C) greater social Connectedness, (H) fostering Hope and optimism, (I) transformation of Identity from one dominated by stigma and a passive patient role, (M) developing new Meaning in life, often deriving meaning from mental health experiences, and (E) Empowerment and responsibility for self-managing mental health (12).

The understanding of processes associated with recovery provides a framework for the development of novel interventions suitable for use in mental health services. This field is at an early stage. Results of the recent REFOCUS trial suggested that delivering training based on the CHIME model to promote recovery-oriented practice in services may have a relatively limited impact on measures of personal recovery (13). On the other hand, positive outcomes for measures of recovery have been found for a number of self-management programs, which include materials on recovery (14-16). Notably, these programs tend to incorporate a strong perspective of learning from shared lived experience, with trialed interventions typically featuring peer co-facilitation and group format delivery, encouraging peer contact and peer-to-peer discussion. Peer-delivered services, peer worker roles, and peer-facilitated interventions have increasingly been a key component of this broader recovery movement (8). In a qualitative metasynthesis of what people find helpful about peer support, peers providing a positive role model, engendering hope, and forming new connections were the key themes (17). Likewise, peer contact is often highlighted as having contributed to recovery in consumer narratives (12), which suggests that hearing directly from others with shared lived experience may be a useful component in recovery-oriented interventions.

In the project reported here [Self-Management and Recovery Technology (SMART)], we developed and piloted a scalable intervention tool suitable for use within mental health services to promote personal recovery. In developing the intervention, we saw potential in creating resources in a digital format that could be incorporated into mental health service consultations using a tablet computer as well as being directly accessible by consumers (18). Initial studies of Internet-based applications with people with persisting psychosis have indicated self-guided use of digital tools to be feasible with this population (19-21). Moreover, the Internet is potentially empowering of people with mental illness in facilitating peer-to-peer connections (22) and is a means of presenting lived experience material in video format that may be useful in portraying positive hopeful views of peers with mental health problems (23). Hence, digital technology offers a number of possibilities for promoting learning from lived experience.

We developed an online intervention tool featuring lived experience accounts of personal recovery as central to a series of modules based on the CHIME framework. This paper presents data on the feasibility and acceptability of using this tool in sessions with a mental health support worker and provides a preliminary examination of outcomes.

MATERIALS AND METHODS

Website Development

The website was developed through parallel processes of end-user consultation; content conceptualization and writing; development of lived experience video materials; and site design. The consultation process included a reference group of seven mental health service consumers with experiences of psychosis who met every 2 weeks during the development phase; a series of focus groups with mental health practitioners from both clinical services and community support services (24), followed by a monthly practitioner reference group; and a further focus group with family carers.

Content Development

The development of content was an iterative process, combining the conceptual framework of CHIME with input from consultations and emerging content from the filming process. The CHIME framework was used to inform the main content themes, presented as modules as follows:

4 . Me, covering topics related to identity including the effects of stigma; personal growth through experiencing mental health problems; and focusing on strengths.

7 . Life, covering topics related to developing new meaning in life including consideration of the personal values that make life meaningful and identifying related goals.

The video material that was central to the site was developed by conducting a series of interviews with persons with lived experience of psychosis using a semi-structured interview derived from the content framework. To develop the interview questions, a working group that encompassed academic, practitioner, and lived experience expertise developed ideas for questions based on topics within the CHIME model for each theme, refined further with reference group input. Questions were developed to draw out the following: (a) how the theme had been relevant to the interviewee’s experience (e.g., the impact of mental illness and associated stigma on how the person saw themselves) and

Selected interviewees were first phoned by the interviewer to discuss what participation would involve and to ensure they had thought through the implications of appearing on film. They were then sent information about what their participation would involve and were provided with interview questions a few days prior to filming. To ensure that interviewees felt in control of the experience, it was made clear that the interviewer wanted them to discuss only things that they felt comfortable with. The interviewer revisited whether they were willing for the interview to be used after it had been completed, with the option of having any material deleted if wished. A film crew of two (camera and audio) performed the filming, using a two-camera set-up to facilitate editing, with the interviewer off screen. The interviewer used the semi-structured interview as a guide, but allowed the interview to deviate when useful material was being generated. Questions could be repeated to be refilmed if needed to improve delivery, or align with the first person experiential style of the interviews. At the end of the filming session, each participant was given the option to review a transcript of their interview prior to giving permission for use. Interviewees were paid for their time.

Theme: Relationships

Topic: Strengthening connections

Briefing. Here the focus is on what people have done in order to develop a sense of connectedness with other people. We are particularly interested in capturing some of the small tilings people can do to develop a sense of connectedness, for example, little things that the person might be able to do today or tomorrow.

How have you gone about building up relationships or a sense of connection?

Could you tell us the things you do that help you feel connected with the community around you?

FIGURE 1 | Example interview question posed to video participants.


From a pool of 20 potential interviewees who responded to advertising, 11 were selected to form a group with diversity in terms of age, gender, ethnicity, employment status, and sexuality. Interviews were conducted until the material elicited was judged to have reached saturation in its coverage of the content domains; each took approximately 1-2 h.

An extended editing process aimed to generate a series of 2-3 min videos featuring a selection of four to six interviewees discussing key issues for each topic. Given that recovery is characterized as highly individual (12), we aimed to capture different experiences and points of view within each video. Interviews were transcribed, and each line coded by a member of the research team into the various video topics. The combined filmed material for each video topic was reviewed by the content group, with excerpts selected that represented the most useful or impactful material obtained while reflecting a range of experiences and perspectives. A total of 26 videos were produced using this method. Additionally, 11 videos were made introducing each of the peers.

In addition to these lived experience videos, five videos of mental health professionals' experiences and two videos of family members' experiences (produced in a similar way) were included as part of material on working with services and relationships. Additionally, 12 videos were produced featuring either a consumer leader or academic expert contributing additional material that elaborated on what was raised in the other videos or by addressing points that had not been captured by the interviews. An introduction to each module was also filmed, featuring a consumer leader as guide. In total 64 videos were included in the final package.

Text content and reflective exercises were added to summarize key points and complement the lived experience content with material from relevant therapeutic approaches (e.g., on implementing coping strategies, on changing health behavior, on identifying personal values).

Website Design

The design of the site was informed by published guidelines for website development for severe mental illness designed to minimize the impact of difficulties in thinking and memory (25), combined with input from the consultation process, and consideration of how the site could be designed in a way that reinforced recovery. The site was optimized for tablet computer and mobile phone use. Navigation was simplified by organizing content in a minimal number of levels (topic, subtopic), making use of touchscreen scrolling to reduce the required number of page loads and having a single constant menu button. Links between pages were clearly labeled, pages were designed to have minimal distractions, and content was developed to be simple, clear, and logically organized. Key design principles derived from the consultations were (a) simplicity of layout and navigation, (b) flexibility in use (e.g., material can be completed in any order), (c) interactivity, (d) catering to different learning styles and preferences by presenting content in multiple ways, (e) access to any information entered being controlled by the consumer, and (f) promoting a positive emotional experience while engaging with the website (24). Consideration of how recovery processes could be facilitated by the design and features of the site included the following: (a) promoting connectedness by allowing users to comment on material and contribute to a user forum on the site and allowing users to share content with workers, family members, and others; (b) promoting the person taking ownership of their identity by allowing personalization of user profiles and customization of content; and (c) promoting empowerment and responsibility in self-management by the ability to track parameters such as sleep and mood and to set and view goals developed from the material.

The site is accessible only by creation of an account, which enables the user to enter information in reflective exercises, charts, and task lists and to select a username and avatar for posting public comments and using the forum. Forums and comment feeds are monitored by the research team to assess risk of harm to participants, and participants are also able to report any offensive comments for moderator review. Example screenshots are shown in Figure 2.

Pilot Study

Design

To examine feasibility and proof-of-concept of using this tool within service delivery, a pilot study was conducted in the form of a single-arm trial of a mental health worker-facilitated intervention using the SMART website on a tablet computer with participants. All participants received the intervention in addition to treatment as usual during a 3-month delivery window. Assessments were completed at baseline and at 3 months. The project was conducted in accordance with the Declaration of Helsinki, and was approved by the Human Research Ethics Committees at The Alfred (study no. 139-14), St Vincent's Hospital Melbourne (study no. 041.14) and Melbourne Health (study no. 2014.087). All participants gave full informed consent prior to commencement.

Participants

Participants were recruited through a combination of mail-outs of consumers, clinician referral at community mental health services, and presentation to consumers at residential services in metropolitan Melbourne. Inclusion criteria were: (a) aged 18-65 years; (b) diagnosis of a psychotic disorder (schizophrenia-related disorder or bipolar disorder or major depressive disorder with the presence of a severe episode with psychotic features within the past 2 years), confirmed using the Structured Clinical Interview for DSM-IV-TR Axis I Disorders [SCID; (26)];

or the commencement or completion of formal psychological therapy within the preceding 2 months.

Procedures

A research assistant met with potential participants to obtain informed consent and complete baseline assessments, including the SCID, WTAR, and first administration of outcome measures. Eligible consenting participants were then provided with a time for their appointment with one of the two facilitators, and went on to complete the eight session intervention. The postintervention assessment was scheduled for 3 months following the baseline assessment, to allow time for missed sessions. The post-intervention assessment also included a treatment evaluation questionnaire. Following this assessment, participants were also contacted by telephone by the project manager to obtain feedback on the site to refine content, during which they were also asked questions about their experience of the intervention more broadly.

Intervention

The intervention consisted of eight 50-min face-to-face sessions with an experienced mental health support worker (facilitator), using a tablet computer from which the SMART website was accessed. Participants were assigned to one of two trained facilitators, seconded from the community mental health support sector, and attended sessions at weekly to fortnightly intervals within a window of 3 months, in addition to the participant’s routine treatment. An account for the participant to use the website was set up during the first intervention session, and they were shown how to access it to facilitate use outside sessions.

Sessions involved the collaborative selection of content from the seven themes on the site, followed by shared viewing and discussion of website material. Discussions included reflecting on the website content as applicable to the participant’s own recovery, considering changes participants may wish to enact based on these reflections, and setting goals for the upcoming week. Facilitators encouraged participants to use the website between sessions, complete reflective exercises, and/or make public posts about the content, if willing.

Feasibility and Acceptability

Feasibility and acceptability were indexed by the following:

Outcome Measures

The following measures were completed pre- and post-intervention to provide a preliminary assessment of the outcome.

Personal Recovery

Personal recovery was the primary outcome. The Questionnaire for the Process of Recovery [QPR; (28)], a 22-item self-report measure developed in conjunction with mental health service users to assess personal recovery. The QPR has good psychometric properties (29) and was used because of its strong alignment with the CHIME framework (30). However, because it was a relatively new measure for which sensitivity to intervention effects was unknown, we also included total score on the more established 41-item Recovery Assessment Scale (31) as a second measure of personal recovery.

Recovery dimensions

Hope was measured using total score on the Schizophrenia Hope Scale (32), a 9-item questionnaire assessing optimism and hope for the future, rated on 3-point items (disagree, agree, strongly agree). Social connectedness was measured using total score on the Friendship Scale (33), comprising six 5-point (not at all to almost always) items.

Psychotic Symptoms

The interviewer-rated Positive and Negative Syndrome Scale [PANSS; (34)] was used to assess the severity of psychotic symptoms and their impact on behavior and functioning. The two research assistants were trained in the standardized administration of the PANSS. In addition, the Subjective Experiences of Psychosis Scale (35) was used to assess the subjective impact of psychotic symptoms in participants reporting ongoing positive symptoms at baseline. This is a 29-item questionnaire on which participants rate the positive and negative impact of their symptoms on aspects of their feelings and behavior, such as “hope for the future” and “ability to socialize.” Subjective impact is rated on a 5-point scale from not at all to very much in the past week. The negative impact subscale score was used in analyses.

Emotional Symptoms

The 21-item Depression Anxiety Stress Scale [DASS-21; (36)] total score and its associated subscales were used to assess emotional symptoms. Participants reported on their experience of symptoms related to depression, anxiety, and stress in the past week, on a 4-point scale from did not apply to me at all to applied to me very much or most of the time.

Quality of Life

Total score on the Assessment of Quality of Life-8 Dimension [AQoL-8D; (37)] was used to assess health-related quality of life. This 35-item questionnaire encompasses eight dimensions of physical and psychosocial health, with lower scores indicating fewer issues related to subjective quality of life in the past week.

Process Measures

It was hypothesised that the lived experience content of the intervention would influence outcome by increasing self-efficacy for positive recovery, and by reducing the extent to which mental illness is viewed in negative stigmatized terms. The proposed mechanism was assessed by the following measures.

Self-Stigma

Self-stigma was used as an index of negative views of illness, measured using the 29-item Internalized Stigma of Mental Illness Scale [ISMI; (38)] which includes subscales of alienation, stereotype endorsement, discrimination experience, social withdrawal, and stigma resistance. Items are rated on a 4-point scale from strongly disagree to strongly agree.

Self-Efficacy

Self-efficacy was measured using total score on the Generalized Self-Efficacy Scale (39), a 10-item questionnaire rated on a 4-point scale from not at all true to exactly true.

Additionally, qualitative feedback on experiences of using the lived experience videos was collated from the post-intervention interview. Use of antipsychotic medication was also recorded at baseline and 3 months as a potential confound.

Treatment Evaluation

Subjective perceptions of the helpfulness or otherwise of the intervention were assessed by the item “Do you feel that using the website made the impact of your mental health problems better, or worse, or no different?” based on an item used by our group in previous trials with this population (40, 41). This was rated on a 5-point scale from much worse to much better. Additionally, 11 5-point Likert items were developed to assess whether participants endorsed changes having occurred in relation to material covered on the site (e.g., “I feel more hopeful about my recovery”). Questions were also included in the qualitative phone interview to gather feedback on use of the site during sessions with a worker, and on specific site elements including the videos.

Statistical Analyses

In this pilot study, the emphasis was on examining acceptability and feasibility, as well as allowing a preliminary estimate of treatment effects, rather than using inferential statistics to hypothesis-test specific outcomes. Correspondingly, effect sizes and confidence intervals were calculated for the mean pre- to post-intervention change score. Standardized effect sizes were calculated by dividing the mean change by the average SD of pre and post-intervention scores or, if variances were unequal, the baseline SD (42). A series of paired t-tests was also conducted to indicate where two-tailed significances fell within p < 0.05. Complete data were available for all participants.

RESULTS

Twelve potential participants were recruited for the study. Two of these were excluded at baseline: one due to a recent medication change and the other due to participation in another research project. Ten participants completed the baseline assessment (nine males; mean age 42.6 years, SD 12.47, range 23-62 years; six single, four divorced). Nine had a diagnosis of schizophrenia, and one of schizoaffective disorder. None worked full time; two were in paid part-time work and one in volunteer work. All were receiving antipsychotic medication. At baseline, seven participants reported using the Internet at least daily, one once per week, and two “rarely or never.” Half of the participants reported that they “rarely or never” used the Internet to access information about mental health.

Feasibility and Acceptability

In-Session Use

The website was used in 76 of 80 sessions that were attended. Of the remainder, two were initial sessions spent assisting participants setting up email to use the site, and one was a final session consolidating the program material without use of the site. There was only one session in which the support worker was not able to use the site with the participant. Session notes indicated that the participant’s engagement had been threatened in a previous appointment, when the facilitator misunderstood something that the participant was discussing, and subsequently directed them to an unrelated topic on the site. The session was spent having a broader discussion of recovery without use of the site to reengage the participant, and the participant and the worker went on to use the site further in their remaining sessions.

Qualitative feedback on the use of the site in-session is collated in Table 1. Responses suggested that the process of integrating the website as a tool in sessions functioned well. Some participants expressed that they would have been less engaged or unable to use the site independently without the facilitator sessions, and a number of participants commented that the site facilitated discussion with the worker.

Between-Session Use

Six of the 10 participants independently logged on to the site outside of sessions, with a median of 4.5 log-ons among those who did this (range 1-14). Among these six participants was one of the participants who “rarely or never” used the Internet at baseline, the remainder being daily users. Six participants posted public comments on the site, with a median of two posts among those who posted (range 1-20 posts).

Drop Out and Satisfaction

All 10 participants attended the full course of eight intervention sessions, and all 10 participants said during the post-intervention interview that they would recommend the site to others.

Emotional Impact of Site Use

No participants indicated a negative emotional impact of use of the site, with all 10 reporting a positive effect of using the site on how they felt (responses better or much better).

TABLE 1 | Participants' feedback about using the site together with a worker.

P1. I think the technology was, it was a guide, it kept our discussion going in a direction that we wanted it to go in. And it would raise the topic or it would raise the, the next discussion, so it was guiding what we were going through. But we did do a lot of talking with [the facilitator] and I felt the two worked together really, really well ... perfect. I did say to [her] though that I felt that the facilitator was needed. I felt that while I was at home I didn't have, there was no accountability. I didn't have anyone looking over my shoulder telling me “You must do that” and “you must go to the website”, “you must do a module or whatever”, there was no accountability. But with a facilitator where I'm going to see [her], you know, within a few days, and we were going to discuss this, then there was accountability, I had to get some things done.

P2. Yeah it was very easy and [the facilitator] explained everything well to me, yeah. And ah, even, I think even just without her I think I could have gone through it myself it would have been quite easy ... yeah, yeah. There was some bits in there, um some bits that were a bit difficult to understand, yeah. Not much, but there were two or three parts that she helped me with . And not only that, ah, having someone there as a support to go through every single one of them, I think that very helpful . I guess the SMART, the website goes into more detail into aspects of my life.

P3. We were able to acknowledge and cover things in more depth than I would have by myself.

P4. It's fine, so the iPad was useful, but I'm not the type of person to sort of sit down and do that sort of stuff. I'm more of an interactive person with whoever I'm talking to. . I enjoyed talking to her more than using the iPad.

P5. I thought it worked well. Yeah. ... Like with the iPad, I thought that, what do you want to discuss today? There's always, it was more, this program, it was more about what you had to say and what you thought of situations instead of, instead of feeling intimidated when you go in to other ways to see a worker or feeling like oh what are going to say and then feeling like intimidated. But in this case I didn't feel intimidated, I knew [the facilitator] well, and I thought that she did a great job just explaining everything to me, patience, and all of it.

P6. I thought it was really easy. Really easy. Smooth and, ah yeah, just a pleasant experience ... . if it was just sort paper and pencils, it sort of got a bit dull after a while. . but the iPad and the website made it quite colourful and a bit more interesting.

P7. Well I didn't use it, I know it's going to sound funny, but we didn't use it much, just for me to get to where I'd written it at home and then read it out to her, “oh this is what I've written and this is what I've written, and this is what I've done,” and then discuss it because I can't type properly on an iPad and I like to type really fast and you know, and be able to check my spelling and everything so I just did it at home., and I was happy. ... I'd just log on and do some stuff at home and then in the sessions they were really just to go over what I'd done at home and what had come up and so she was sort of acting therapist, poor [facilitator]. ... No with the iPad, because I got to share all, everything I'd written, and we'd talk about what I wrote. Talk about subjects and say, “What subject should I do next?” and “What's that involve?” And, “Maybe this one would work,” and then I'd say, “Can you do a print out of the PDF, and yadayadayada.” A lot more involved than just seeing a therapist, you know what I mean.

P8. [Without the website] we wouldn't have had nearly as much to talk about. And then I would have been more stuck for words I think. I wouldn't have been able to talk about all the issues that we had discussed about the website so it would have been a bit more difficult I think.

P9. It was just really good. A good experience with [the facilitator] and the iPad.

P10. I would've really hated it if I did it by myself, because I probably wouldn't have got there anyway, I mean anywhere, but that's why I thought, I didn't mind it so much, because um people like [the facilitator] were just such a good guide. But something like, with computers, I couldn't do it myself, even though it seemed pretty simple, once [the facilitator] was showing me what to do, I just said, I really would hate to do it by myself .

Outcomes

Estimated effect sizes on the outcome measures are presented in Table 2. On the primary outcome of personal recovery, an estimated medium effect size was observed on the QPR, which in spite of the small sample size was statistically significant. A similar magnitude effect was observed on the RAS as a second measure of recovery (also statistically significant). Among other outcomes, medium effects were estimated for the subjective negative impact of psychosis symptoms, emotional symptoms on the DASS, and hope, but there was a negligible effect on social connectedness. A small effect size was estimated on the PANSS and the AQoL-8D.

Among process measures, a small to medium effect was estimated on self-stigma, but negligible effects were evident on self-efficacy. Examination of subscales of the ISMI self-stigma measure suggested that the strongest effects were in reducing perceived alienation with an estimated moderate to large effect size (statistically significant), while the estimated effect on other domains, such as negative stereotype endorsement, were negligible. Two participants had increases and three had decreases in antipsychotic medication dose during the trial, but changes on the personal recovery measures were not correlated with changes in chlorpromazine-equivalent dose.

In response to the question “Do you feel that using the website made the impact of your mental health problems better, or worse, or no different?” eight participants reported that their mental health was better or much better, with the remaining two reporting it was no different. Agreement was also high on all items of the treatment evaluation questionnaire (Table 3). Specific feedback on the lived experience videos is collated in Table 4.

DISCUSSION

This study examined the feasibility of an intervention targeting personal recovery in psychosis, a domain for which intervention development is a priority. It involved the novel combination of lived experience-based content on recovery, presentation via a digital medium, and delivery integrated with face-to-face mental health sessions. Overall, it appeared feasible to deliver an intervention in this way, and there were promising findings on the primary outcome of personal recovery.

TABLE 3 | Participant responses on the treatment evaluation questionnaire.

Since starting SMART ...

Number agreeing or strongly agreeing (N = 10)

I understand more about my mental health

8

I feel more connected with people

7

I feel more hopeful about my recovery

9

I have progressed in my personal recovery

9

I have a stronger sense of my identity

8

I have a better idea about what my values are

8

I feel more confident about making plans

8

I feel more confident about my rights

8

I feel more confident about working with services

9

I feel more confident about managing my stress

8

I feel empowered to improve my physical and

10

mental health

TABLE 2 | Estimated effects on outcome measures.

Measure

Mean (SD)

Change score

Effect size

P

Pre

Post

Mean

95% CI

d

95% CI

Personal recovery

QPR

57.50 (11.65)

62.90 (11.89)

5.40

[0.87, 9.93]

0.46

[0.07, 0.84]

0.024

RAS

154.10 (13.59)

163.20 (18.80)

9.10

[1.44, 16.76]

0.56

[0.09, 1.04]

0.025

Recovery dimensions

SHS

17.60 (3.92)

19.80 (6.41)

2.20

[-0.515, 4.92]

0.56

[-0.13, 1.25]

0.10

Friendship Scale

16.00 (2.87)

16.22 (4.66)

0.22

[-3.25, 3.70]

0.08

[-1.13, 1.29]

0.89

Psychotic symptoms

PANSS total

65.70 (18.58)

61.40 (19.51)

-4.30

[-12.51,3.91]

-0.23

[-0.66, 0.21]

0.27

PANSS positive

17.90 (8.05)

15.70 (5.87)

-2.20

[-4.65, 0.25]

-0.32

[-0.69, 0.03]

0.07

PANSS negative

14.50 (3.60)

14.80 (7.33)

0.30

[-4.42, 5.02]

0.06

[-0.81,0.91]

0.89

PANSS general

33.30 (10.12)

32.90 (9.43)

-0.40

[-3.91,3.10]

0.04

[-0.40, 0.32]

0.80

SEPS negative impact

80.43 (30.84)

61.14 (18.87)

-19.29

[-43.82, 5.25]

-0.78

[-1.76, 0.21]

0.10

Emotional symptoms

DASS total

25.20 (16.71)

16.90 (11.21)

-8.30

[-16.91,0.31]

-0.60

[-1.21,0.02]

0.06

DASS depression

8.60 (5.72)

6.50 (5.40)

-2.10

[-6.46, 2.26]

-0.38

[-1.16, 0.41]

0.31

DASS anxiety

7.60 (5.19)

5.20 (2.57)

-2.40

[-5.32, 0.52]

-0.46

[-1.03, 0.10]

0.10

DASS stress

9.00 (6.60)

5.20 (4.59)

-3.80

[-6.79, -0.81]

-0.68

[-1.21, -0.14]

0.018

Quality of life

AQoL-8D total

88.20 (17.25)

83.90 (16.58)

-4.30

[-12.01,3.41]

-0.25

[-0.71,0.20]

0.24

Process measures

GSES total

27.80 (3.46)

28.40 (5.44)

0.60

[-2.36, 3.56]

0.13

[-0.53, 0.80]

0.66

ISMI total

65.70 (13.48)

61.20 (13.60)

-4.50

[-9.28, 0.28]

-0.33

[-0.69, 0.02]

0.06

ISMI alienation

15.70 (3.89)

13.20 (3.33)

-2.50

[-4.09, -0.91]

-0.69

[-1.13, -0.25]

0.006

ISMI stereotype endorsement

11.80 (3.52)

11.60 (3.57)

-0.20

[-1.14, 0.74]

-0.06

[-0.32, 0.21]

0.64

ISMI discrimination

12.00 (3.23)

11.50 (3.47)

-0.50

[-1.53, 0.53]

-0.15

[-0.46, 0.16]

0.30

ISMI social

15.00 (3.74)

14.40 (3.75)

-0.60

[-2.15, 0.95]

-0.16

[-0.58, 0.25]

0.41

ISMI stigma

11.20 (2.10)

10.50 (1.72)

-0.70

[-2.13, 0.73]

-0.37

[-1.12, 0.38]

0.30

N = 10, except for SEPS negative impact (N = 7).

QPR, Questionnaire for the Process of Recovery; RAS, Recovery Assessment Scale; SHS, Schizophrenia Hope Scale; PANSS, Positive and Negative Syndrome Scale; SEPS, Subjective Experience of Psychosis Scale; DASS-21, Depression Anxiety Stress Scale 21; AQoL-8D, Assessment of Quality of Life-8 dimension; GSES, Generalized Self-Efficacy Scale; ISMI, Internalized Stigma of Mental Illness.


Our attempts to develop the main content of the site by using a process of editing together various lived experience interviews showed this to be a feasible approach. We used 11 lived experience speakers to provide diversity in age, gender, ethnicity, and sexuality within the group, which was sufficient to produce material for all topics. Not all content within the topics was covered in this way, but complementary scripted videos from experts and text material was used to complete an effective website in which the dominant content was explicitly authored by peers. While many online interventions include “client perspective” videos to illustrate other material [e.g., Ref. (43, 44)], this is the first online intervention, we are aware of, that has an explicit focus on lived experience material as the main vehicle for change.

Use of the site in-session appeared to be feasible and acceptable to participants. The website was used regularly in-session, no participants dropped out, and participants gave generally positive feedback about how the use of the site integrated with face-to-face work. It appeared from participant feedback that many would have found it harder to use or maintain engagement with the site had it not been integrated with sessions with the facilitator. The feedback is consistent with the broader digital mental health literature, where therapist-assisted interventions tend to be engaged with for longer than self-guided interventions (45). However, lower levels of use of digital technology (46) and higher rates of disability among persons with severe mental illness suggest some people may be more reliant upon support to utilize online materials. A blended approach offers a means of capitalizing

TABLE 4 | Participants’ feedback about lived experience videos.

P1. I enjoyed, I guess, what's the name of the word, the reinforcement, or, seeing somebody else going through the same situation with the same feelings, somebody I could relate to, I found, it was something I hadn't been through before and, and that made me feel good, that felt great and a lot more at ease from watching the video. The video also ... what I did try and do once or twice was to answer questions without watching the video and then, going back watching the video, realised that the video was actually opening up the scope, it was actually scoping out the area ahead of what the questions were going to be like.

P2. The fact that others are sharing their own experience. ... Yeah, and I look at the video and even though I was hospitalised before, when I come out - it's been a while since I came out - I forget that ah, I'm not alone. Yeah.

P3. Really enlightening. Made me feel like I am not alone.

P4. Well ... relate to people ... what they're saying: this is what happened to me and how I got over it, and what I did. Yeah.

P5. I like the fact that everyone's so different, it's so, like they all have different, and they're all unique, and they all had good things to say. Like what I mean by good is, you know, relevant to people with like, yeah. I didn't feel so alone. So that was a good thing. ... I just felt like I could relate to someone. I wasn't so alone.

P6. There were obviously the different individuals who explained their scenario and talked about each topic in the video, and then said what that topics means to them, and how certain questions around the topic are answered, and it was all good, it was all insightful. ... Yes, the videos were really good, they were organised, they were structured, quite informative, honest, and um yeah, so like lots of multi-perspectives on topics - yeah, that was good.

P7. Because I could just sit there and watch a whole half hour of them talking and get so inspired, and so moved, you know. ... it makes you realise you're not alone, and that you're not some frumpy sort of, the bad image of mental illness: not washed, not clothed well, smells bad, can't coherently keep a sentence together, looks off into the distance, is aggressive or threatening or sullen. You know what I mean?

P8. I could relate to a lot of peoples' stories, and they had a similar experience to mine, so I thought that was good. And then I answered a few questions and sent a few comments to [the facilitator] and that sort of thing. So yeah I just, I gained more insight into my condition I think. I've always had a lot of insight, but just hearing other peoples' experiences; when you think you've got your own mind made up about your illness and you won't listen to anybody about your illness, and you need to think “oh okay,” you think you're right, but there are a lot of other people who have varying symptoms, and it was just good to hear other peoples' opinions and impressions of their own diagnosis, and that sort of thing; what they do to tackle their problems. So it was good.

P9. I can relate to some of the things they were talking about in my own life, and it just makes me more aware and more determined to overcome the obstacles that I've been facing.

P10. They had people talking about how to handle stress, and I put my feet in their shoes and sort of could understand, you know, um where those people were coming from, their experiences.

on the scalability of digital interventions to deliver quality structured interventions, while bearing in mind the barriers to independent Internet use experienced by this group. Indeed, although most participants used the site between sessions as well, which was encouraged, not all of the participants in this pilot did so, suggesting that a significant proportion of people in this population would be reliant on in-session use. Indeed, it should also be noted that most of our sample were daily Internet users at baseline, so their existing levels of computer use were higher than average for this population (46). Given our vision for the website as a vehicle to facilitate discussion between the consumer and worker about recovery, the modest between-session use was not problematic. A number of participants' responses confirmed that use of the site did help them discuss issues which otherwise might not have been raised or would have been difficult to raise.

Video-based tools may have benefits beyond the present aim of facilitating discussions about recovery: future research could investigate whether patient-worker interactions around embarrassing or sensitive topics (e.g., discussing ambivalence about medication) are supported by tools of this kind. The technology may have broader applications in practice, such as in promoting supported decision making [e.g., Ref. (47)], or as a tool for formal psychological therapies.

As a preliminary proof-of-concept study with a small sample, analysis of outcome was not designed to hypothesis-test efficacy, but to establish whether estimated effects were in a range suggesting full scale trialing to be worthwhile. While we cannot be certain that other ongoing interventions had no impact on outcomes, results were promising, with a moderate effect size being estimated on both measures of recovery that were used. Feedback from participants was also consistent with the intervention having a beneficial impact upon recovery, and participant feedback additionally identified no negative effects. The estimated effect size on recovery is similar to effects observed for other psychosocial interventions for persisting psychosis (48). Together, these findings suggest value in conducting a larger-scale controlled trial of this intervention (49).

AUTHOR CONTRIBUTIONS

All authors participated in the development and pilot of the digital intervention. NT, FF, NL, JF, EL, CN, SF, LS, and GM designed the website and developed its content. NT, JF, FF, RF, and TS developed the therapist intervention protocol. NT, K-AV, and FF conducted analyses and prepared the manuscript. All the authors read and approved the final manuscript.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the contributions of the video contributors; Rybazoid (video production); Fidget Friend (software development); Alfred Health, Mind Australia, Northwestern Mental Health, St Vincent's Hospital Melbourne, Tandem, VICSERV, the Victorian Mental Illness Awareness Council, Wellways Australia (Mental Illness Fellowship); Katrina Lindblom, Friederike Wahl, Robert Shaw, Claire Young, Jo-Anne Abbott, Alex Lopez Lorca, Sonja Pedel, Maheswaree Kissoon Curumsing, and members of the focus and reference groups.

REFERENCES

FUNDING

This study was funded by the State Government of Victoria Department of Health Mental Illness Research Fund (MIRF33). The funder had no role in the design of the study or reporting of results.

Robertson S, et al., editors. Proceedings of the 24th TheMHS Mental Health Services Conference 2014. Balmain, NSW: TheMHS (2015a).

Conflict of Interest Statement: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Thomas, Farhall, Foley, Leitan, Villagonzalo, Ladd, Nunan, Farnan, Frankish, Smark, Rossell, Sterling, Murray, Castle and Kyrios. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Original Paper

A Web-Based Tool to Support Shared Decision Making for People With a Psychotic Disorder: Randomized Controlled Trial and Process Evaluation

Lian van der Krieke1, MSc, MA; Ando C Emerencia2, MSc; Nynke Boonstra3, PhD; Lex Wunderink1,3, PhD; Peter de Jonge1, PhD; Sjoerd Sytema1, PhD

1University of Groningen, University Medical Center, University Center for Psychiatry, Groningen, Netherlands

2University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science, Groningen, Netherlands

3Friesland Mental Health Care Service, Leeuwarden, Netherlands

Corresponding Author:

Lian van der Krieke, MSc, MA

University of Groningen

University Medical Center

University Center for Psychiatry

Hanzeplein 1

Groningen, 9700 RB

Netherlands

Phone: 31 503612108

Fax: 31 503619722

Email: j.a.j.van.der.krieke@umcg.nl

Abstract

Background: Mental health policy makers encourage the development of electronic decision aids to increase patient participation in medical decision making. Evidence is needed to determine whether these decision aids are helpful in clinical practice and whether they lead to increased patient involvement and better outcomes.

Objective: This study reports the outcome of a randomized controlled trial and process evaluation of a Web-based intervention to facilitate shared decision making for people with psychotic disorders.

Methods: The study was carried out in a Dutch mental health institution. Patients were recruited from 2 outpatient teams for patients with psychosis (N=250). Patients in the intervention condition (n=124) were provided an account to access a Web-based information and decision tool aimed to support patients in acquiring an overview of their needs and appropriate treatment options provided by their mental health care organization. Patients were given the opportunity to use the Web-based tool either on their own (at their home computer or at a computer of the service) or with the support of an assistant. Patients in the control group received care as usual (n=126). Half of the patients in the sample were patients experiencing a first episode of psychosis; the other half were patients with a chronic psychosis. Primary outcome was patient-perceived involvement in medical decision making, measured with the Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE). Process evaluation consisted of questionnaire-based surveys, open interviews, and researcher observation.

Results: In all, 73 patients completed the follow-up measurement and were included in the final analysis (response rate 29.2%). More than one-third (48/124, 38.7%) of the patients who were provided access to the Web-based decision aid used it, and most used its full functionality. No differences were found between the intervention and control conditions on perceived involvement in medical decision making (COMRADE satisfaction with communication: F168=0.422, P=.52; COMRADE confidence in decision: F167=0.086, P=.77). In addition, results of the process evaluation suggest that the intervention did not optimally fit in with routine practice of the participating teams.

Conclusions: The development of electronic decision aids to facilitate shared medical decision making is encouraged and many people with a psychotic disorder can work with them. This holds for both first-episode patients and long-term care patients, although the latter group might need more assistance. However, results of this paper could not support the assumption that the use of electronic decision aids increases patient involvement in medical decision making. This may be because of weak implementation of the study protocol and a low response rate.

http://www.jmir.org/2013/10/e216/

XSL-FO

RenderX

Trial Registration:       Dutch Trial Register (NTR) trial number:   10340;

http://www.trialregister.nl/trialreg/admin/rctsearch.asp?Term=10340         (Archived by WebCite at

http://www.webcitation.org/6Jj5umAeS).

(JMed Internet Res 2013;15(10):e216) doi:10.2196/jmir,2851

KEYWORDS

psychotic disorders, schizophrenia; computers; computer-assisted decision making; shared decision making; feasibility studies, randomized clinical trial

Introduction

Shared decision making in mental health care has been dubbed an ethical imperative [1]. Since the rise of recovery-oriented medicine, patients have been acknowledged as experiential experts and equal partners in communication with clinicians. Research has shown that people with severe and persistent mental disorders are no exception. People with psychotic disorders are able and willing to participate in medical decision making [2,3]. However, the desire for participation is greater than the amount of participation they actually experience [4,5]. A range of obstacles hamper successful implementation. Most clinicians believe in the benefits of shared decision making, but time constraints and a large number of clinical responsibilities prevent them from practicing it [6,7]. Moreover, patients may not be used to actively participating in medical decision making and they can lack access to medical information that is easily intelligible [8].

Drake and Deegan [9] stressed the need for decision aids and support centers to ensure the development of an infrastructure that facilitates the practice of shared decision making. Several initiatives have been developed in this area. For instance, in Germany, Hamann et al [3] investigated the effectiveness of a shared decision-making intervention with a printed decision aid for inpatients with schizophrenia. They found that patients using the decision aid had better knowledge about their disease and had a higher perceived involvement in medical decisions compared to a control group that received care as usual [3]. Recently, a special case was made for electronic decision aids [10] because they have various advantages over paper-based decision aids, such as presenting personalized information based on smart algorithms. So far, 3 electronic decision aids have been developed and investigated to support shared decision making in the treatment planning for people with severe mental disorders, but the results are inconsistent [11,12]. A pilot study by Deegan et al [11] showed that outpatients were able to work with a Web-based program to support shared decision making in psychopharmacological consultation. Patients used the program on computers at the clinic where experiential experts were available for assistance. Two small-scale randomized clinical trials were conducted [12,13]. The first trial showed that patients were able to electronically design their own care plan, but there was no difference between intervention and control groups in satisfaction with the care planning process, which was the primary outcome [12]. The second trial reported that a Web-based support system encouraging patients to discuss their current status and treatment with their clinician resulted in patients being more verbally active during health visits [13]. More evidence is needed to determine whether electronic decision aids are helpful in clinical practice and can lead to increased patient involvement and better outcomes. In addition, more information is needed about what proportion of patients are willing and able to work with Web-based decision aids and in what form (with or without assistance, using their own computer or a clinic computer). This paper reports on a randomized controlled trial and process evaluation of a Web-based intervention to facilitate shared decision making, with or without assistance, for people with psychotic disorders. Our aim was to investigate this intervention in a naturalistic setting, meaning that all eligible patients were included to be able to determine how many of them would actually use the decision aid.

Methods

Ethical Considerations

Informed consent was obtained by research nurses. Patients were provided with an information brochure and they received a phone number and email address of a research assistant who they could contact for further information. The national Dutch medical ethical committee for mental health care (Medisch-ethische Toetsingscommissie instellingen Geestelijke Gezondheidszorg; METiGG) assessed the study protocol and judged that the study could be conducted without the committee’s approval. The trial was registered at the Dutch Trial register (NTR trial number: 10340).

Setting and Participants

The study was carried out in a Dutch mental health institution (Friesland Mental Health Care Service, city of Leeuwarden) with a catchment area of approximately 650,000 inhabitants. Data were collected from June 2011 to July 2012. The trial was completed when all patients provided their last measurement. Patients were recruited from 2 outpatient teams for psychosis: the early intervention for psychosis team (a multidisciplinary team for the treatment of patients with a first episode of psychosis) and a rehabilitation team (a multidisciplinary team for patients with chronic schizophrenia). We used broad inclusion criteria. Participants had to meet criteria for a nonaffective psychosis (brief psychotic disorder, schizophreniform disorder, schizoaffective disorder, schizophrenia, or psychotic disorder not otherwise specified) as defined by the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Text Revision) (DSM-IV-TR), be between age 21 and 65 years, and be fluent in Dutch. Participating professionals were all clinicians involved in the care for those patients describe previously (psychiatrists, community psychiatric nurses, psychologists). Internet or computer literacy was not part of the inclusion criteria.

To calculate the sample size, we used the SPSS SamplePower software program (IBM Corp, Armonk, NY, USA). Given an alpha of .05, a power of .80, and an effect size of .50 (based on results of a comparable study [3]), we needed n=64 per group. Because we expected a considerable amount of dropout (50%) and we wanted to investigate what proportion of patients in the participating teams would use the Web-based decision aid, we decided to include all eligible patients treated by the participating teams.

Study Design

We conducted an open-label, 2-group, parallel, randomized controlled trial with approximately the same number of patients in each group. Patients were allocated to either an intervention group that was offered a Web-based tool to support shared decision making or a control group that received care as usual. Randomization of patients was conducted by using the online Research Randomizer [14]. We used block randomization in blocks of 8 (numbers 1 to 4 were considered intervention condition; 5 to 8 control condition). A research assistant located at the mental health institution participating in the study created a spreadsheet file listing all participants in ascending order by research number. Another research assistant located at our research center added the randomization conditions to the spreadsheet, assigning participants to the interventions.

Treatment Conditions

Control Condition

Patients in the control condition received care as usual, as described in the local disease management program for the treatment of people with psychosis. Treatment modules were initially chosen by a clinician in accordance with a treatment path that a patient entered based on the staging of the disorder (first episode or stabilizing/rehabilitation phase), clinician-rated scores on the Health of the Nation Outcome Scale (HoNOS), and patient-rated scores on the Camberwell Assessment of Need Short Appraisal Schedule (CANSAS-P). During a treatment plan meeting, clinicians informed patients about the indicated treatment modules and also discussed alternatives. A final decision was made in a process of shared decision making (which was not further specified in the disease management program).

Intervention Condition

Patients in the intervention condition received care as described in the local disease management program for the treatment of people with psychosis plus they were offered the opportunity to make use of the Web-based information and decision tool (see Multimedia Appendix 1). This tool is meant to support patients in acquiring an overview of their care needs and of the treatment modules provided by their mental health care organization. The tool functions as a website consisting of 3 webpages and a home page. The home page briefly explains the aim and procedure of the website. The first webpage presents a questionnaire about care needs based on items of the CANSAS-P (see Figure 1). The second webpage offers a digital catalog with descriptions of treatment modules dynamically linked to the outcomes of the questionnaire in the first webpage (see Figure 2). For instance, a reported need for more information about symptoms and medication use was linked to information in a module about psychoeducation, whereas a reported need on items about living a meaningful life and doubts about the future was linked to a module about loss and longing.

In addition to this selection of modules, patients also had the opportunity to view all available treatment modules irrespective of the questionnaire outcomes. The information about the available modules in the catalog included an overview of its content and duration; a description of problems/symptoms the treatment module is usually indicated for; names, functions, and pictures of clinicians involved; a short story by a patient who tells his/her experience with the treatment module (see Figure 3); and, if available, a brief interview with a clinician who tells about his/her experience with the treatment module (advantages, disadvantages, motivation to provide the treatment, etc). The third webpage presents a list of all treatment modules in a checkbox format. The content and design of this Web-based tool was based on an earlier usability study and needs assessment [15]. During the development process, the content of the tool was validated by clinicians and patients. This content was frozen during the trial.

Patients using the Web-based tool were asked to look through the treatment modules and choose the modules of their preference by ticking the appropriate checkboxes. Patients could print the checkbox form and take it with them to their treatment plan evaluation session to discuss with their clinician.

Patients were informed about the Web-based decision aid by research nurses during a biyearly appointment for Routine Outcome Monitoring (ROM), and they were offered an information brochure. Patients were given the opportunity to use the decision aid either on their own (at their home computer, or at a computer of the service) or with support of an assistant. Furthermore, an assistant was available by phone for help on 3 days each week. Patients received a log-in account by email or on paper from an assistant. No further instructions were given about the optimal timing of frequency regarding the use of the decision aid.


Procedure

After randomization, baseline measurement took place during a biyearly face-to-face ROM session for all participating patients. Participating clinicians were asked to complete an attitude questionnaire around the same time. Up to 6 weeks after the ROM session, patients in the intervention condition had the opportunity to make use of the Web-based tool. Approximately 6 weeks after ROM, a meeting was planned between the patient and a key clinician in which ROM results were evaluated and a new treatment plan was created or an existing one was adjusted. Patients were sent a final questionnaire by mail. Upon returning the questionnaire to our research center, they received a gift certificate worth €7.50. We deviated from the procedure described in the original research protocol in 1 important aspect: we conducted 1 follow-up measurement instead of 2 because a second follow-up meeting appeared to be not feasible within the time limits.

Measures

Baseline

Self-reported quality of life was measured with the Manchester Short Assessment of Quality of Life (MANSA) [16]. Patients rate their satisfaction with life on different life domains, in 16 items on a 7-point Likert scale, ranging from very dissatisfied to very satisfied. Higher scores indicate a better quality of life. Psychosocial functioning was measured with the HoNOS [17]. Clinicians rate patients on 12 domains on a 5-point severity scale ranging from no problem to severe or very severe problem. Lower scores indicate a better psychosocial functioning.

Symptom severity was measured with the Positive and Negative Syndrome Scale (PANSS) [18]. Clinicians rate patients during an interview on 7 items about positive symptoms, 7 items about negative symptoms, and 16 items about general psychopathology on a 7-point Likert scale ranging from absent to extreme. Lower scores indicate less symptom severity.

Patients’ preference to participate in medical decision making was measured by the decision-making preference subscale of the Autonomy Preference Index (API) [19]. Patients rate their preference on a 6-item scale in which item scores range from completely disagree (score 0) to completely agree (score 100). A higher score indicates more preference for autonomy.

Outcome

The primary outcome measure was patient-perceived involvement in medical decisions measured with the patient-rated Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE) [20]. The COMRADE consists of 2 subscales, satisfaction with communication and confidence in decision, comprising 20 items in total and scored on a 5-point scale. Higher scores indicate higher perceived involvement.

We used the patient-rated Client Satisfaction Questionnaire (CSQ) [21] as a secondary outcome measurement. The CSQ used in this study consists of 9 items, scored on a 4-point scale. Higher scores indicate higher satisfaction. For the intervention group, we added 6 questions about satisfaction with the Web-based decision tool.

Analysis

Descriptive statistics were used to investigate client characteristics. Baseline measures of both conditions were compared using unpaired t tests or chi-square tests. Difference between the intervention and the control condition on the primary outcome measure was examined using a general linear model with adjustments for patient age and partner status (having a partner yes/no).

Process Evaluation

The intervention described previously can be considered a complex intervention because it consists of several components (use of new technology, implementation in regular care, evaluation) and is highly dependent on the context in which it is delivered. Complex interventions are interventions that contain various interacting components of which the whole is more than the sum of its parts [22,23]. For these interventions, a randomized controlled trial needs to be supplemented by a process evaluation to evaluate their effect. Process evaluations explore implementation issues and contextual factors within the trial. They help to distinguish between ineffective interventions (failure of intervention) and badly delivered interventions (implementation failure) [22].

The process evaluation of this study consisted of

This process evaluation provided data to shed light on how well the intervention was implemented, to what extent the trial outcomes were related to the quality of the implementation and the setting in which it was implemented, and what processes might have mediated these relations.

Results

Process Evaluation

In the process evaluation, we collected data to answer 5 questions about potential problems related to implementation and context.

The first question was: Could the outcomes be affected by a negative attitude of clinicians toward shared decision making or the Web-based decision aid? In a questionnaire-based survey, clinicians’ attitudes were investigated. On a 5-point Likert scale ranging from completely disagree to completely agree, clinicians agreed or completely agreed with 4 statements about shared decision making in general, and 9 statements about the use of a decision aid in decision-making processes. The mean total score on this scale was 3.52 (SD 0.49), meaning that most clinicians showed a positive attitude toward shared decision making and the use of decision aids. Table 1 shows to what extent clinicians agreed or disagreed with the statements.

The second question was: Do clinicians think there are too many hampering factors to realize a process of shared decision making? In addition, 18 clinicians reported that in processes of shared decision making, the following factors were often or almost always experienced as hampering decision making: patients receive contradictory advice from multiple clinicians (12/18, 67%), patients have difficulty accepting their diagnosis (12/18, 67%), and patients are indecisive (10/17, 59%). The following factors were reported as never or sometimes hampering: patients want to participate to a greater degree than the clinician prefers (15/18, 83%), patients have other interfering health problems (15/18, 83%), lack of time (14/18, 78%), cultural differences (14/18, 78%), patients bring in too much information to discuss (13/18, 72%), patients ask for a treatment that is not evidence-based (12/17, 71%), clinician has too little information to make a decision (12/17, 71%), patients do not understand the information (12/18, 67%), patients are too anxious or worried to listen to what the clinician has to say to them (11/18, 61%), and patients refuse treatment that could benefit them (10/18, 56%).

The third question was: Could the outcomes be affected by the clinicians’ judgment about patients’ capabilities and interests? Clinicians were asked to what extent they considered patients to be capable and interested in shared decision making. Of the 128 patient observations, clinicians rated most patients as being able to understand the arguments presented, being capable of making reasonable decisions, and being interested in the topics discussed as well as in participating in medical decision making. Patients who were rated by their clinicians as not capable of making decisions (score 1-3) had a significantly lower score than patients rated as capable of making decisions on both subscales of the COMRADE (COMRADE satisfaction with communication: t48=-3.857, P<.001; COMRADE confidence in decision: t47=-2.368, P=.02. This means that patients who perceived their involvement in medical decision making to be low were judged by clinicians to be less capable of participating in decision making.

The fourth question was: Could any problems be observed with fulfillment of the study protocol? Through researcher observation, several recurring themes were identified during clinical meetings in which the trial was discussed. Case managers sometimes were hesitant and felt troubled to invite intervention patients to make use of the decision tool. First, they were doubtful whether patients were able to handle either the computer program or participation in a research trial. Second, they were not sure that patients would benefit from the decision aid because not all treatment options included in the decision aid were actually offered by their organization (eg, music therapy was listed among the treatment options, but no music therapy was currently offered because of absence of a music therapist). In addition, various clinicians reported that they were unsure when to discuss outcomes of the decision aid with their patients because not all conducted a formal treatment evaluation session with their patients following their ROM assessment. Some only discussed ROM results within the clinical team and not directly with patients.

The fifth question was: Did patients experience any problems with the intervention that was not covered in the satisfaction questionnaire? Open interviews among patients who chose to use or not use the website provided some additional details on the process. First, all patients were initially informed about the decision aid by an information booklet and in a meeting with a research nurse, but most of them received additional explanation from their case manager. Some framed the decision aid predominantly within a research context (“by using the decision aid, you contribute to research”), whereas others described it as an attempt to improve services (“using the decision aid might help you reflect on the treatment you want”). This might have affected patients’ expectations of the intervention. Moreover, interviews revealed discrepancies between the policy of the local disease management program and patients’ experiences in clinical practice. Most of the interviewed patients could not remember their ROM results being discussed with them and some could not remember whether a treatment plan was created.

Allocation and Reception of Intervention

A total of 250 patients (n=124 intervention vs n=126 control) were included in the trial of whom 73 completed the follow-up measurement and were included in the final analysis (response rate 29.2%). Of these 73 patients, 40 were in the intervention and 33 in the control condition. Of the 40 patients in the intervention condition who completed the follow-up measurement, 30 used the decision aid. A detailed overview of the flow of participants is presented in Figure 4.

Table 1. Percentage of clinicians (completely) agreeing with statements about shared decision making and decision aids (n=19).

Item

Agree or completely agree, n (%)

A decision aid will cause patients to ask more questions than they would otherwise have asked

16 (84)

A decision aid will cause patients to be more involved in decision making about treatment3

15(83)

All eligible patients should be invited to use the decision aid

15 (79)

Knowing risks and benefits, most patients want to decide how acceptable treatment is to them

13 (68)

Patients using a decision aid will be much better informed

13 (68)

Patients should see a decision aid before a treatment decision is made

12 (63)

Patients usually want to be an equal partner with physicians in making important treatment decisions

10(53)

With a decision aid, I will be able to reduce time spent educating patients about treatment1

7 (39)

Most patients prefer the clinicians to take responsibility for their medical problems

4 (21)

Using a decision aid will reduce the risk of malpractice

4 (21)

A decision aid will eliminate the need for third-party utilization as second opinion

3 (16)

A decision aid may cause some patients to make the wrong choice

3 (16)

The majority of patients do not wish to be involved in decision making about their treatment

1 (5)

an=18.


Demographic Variables and Baseline Data

Demographic variables and baseline data of patients included in the analysis are presented in Table 2. Patients in the 2 conditions did not differ in age, Global Assessment of Functioning (GAF), MANSA, HoNOS, PANSS, API, level of education, whether they had a job or were studying, and whether or not they used antipsychotics. However, in the intervention group were fewer females (P=.01) and fewer patients with a partner (P=.01).

The patients who dropped out of the study and did not complete the follow-up measurement were slightly younger (t213=-2.129, P=.03) and were more often men (%21=5.6, P=.02) than the patients who did complete the outcome measurement. They did not differ on any of the other baseline characteristics. Patients in the intervention condition who received the allocated intervention versus those who did not receive the intervention did not differ on all baseline characteristics.

Table 2. Demographic variables and baseline data of study participants.

Variable

Intervention (n=40)

Control (n=33)

P a

Age (years), mean (SD)

37 (12.35)

40 (13.47)

.35

Sex (female), n (%)

13 (33)

21 (64)

.01

Education (> 12 years), n

10 (n=12)

10(n=12)

.99

Job or study, n (%)

13 (33; n=39)

16 (48)

.23

Partner, n (%)

9 (23; n=39)

18 (55)

.01

Use of antipsychotics, n (%)

29 (73)

22 (67)

.60

Test, mean (SD) b

GAF

61.8 (9.08)

57.4 (10.91)

.06

MANSA

60.7 (9.50)

62.3 (13.26)

.58

HoNOS

7.7 (4.75)

8.4 (4.32)

.53

PANSS total score

13.3 (5.24)

15.4 (5.51)

.13

API

55.7 (12.72)

52.7 (12.96)

.38

Number of patients from the first episode of psychosis team

16 (40)

13 (39)

.99

within condition, n (%)

aUsing Fisher exact test or t test.

bGAF: Global Assessment of Functioning; MANSA: Manchester Short Assessment of Quality of Life; HoNOS: Health of the Nation Outcome Scales; PANSS: Positive and Negative Syndrome Scale; API: Autonomy Preference Index.

Patient Involvement in Treatment Planning and Their Satisfaction With Care

Intention-to-treat analyses showed that patients in the intervention condition did not differ from patients in the control condition in their perceived involvement in medical decision making (COMRADE) after they had used the Web-based decision aid (COMRADE satisfaction with communication: F168=0.422, P=.52; COMRADE confidence in decision: Fb67=0.086, P=.77; see also Table 3). This was the primary outcome measure. Patients also did not differ in self-reported satisfaction with care (CSQ) (F170=0.014, P=.91).

Per protocol analyses also showed that patients in the intervention condition who received the allocated intervention and completed the follow-up measure (n=30) did not differ regarding their perceived involvement in medical decision making and in satisfaction with care from patients in the control condition (n=33) (COMRADE satisfaction with communication: F157=0.155, P=.70; COMRADE confidence in decision: F156=0.413, P=.52; CSQ: F160=0.789, P=.34).

In an additional analysis, patients in the intervention condition who received the allocated intervention (n=30) were compared to patients in the intervention condition who did not receive the allocated intervention (n=10). No differences were found for patients’ perceived involvement in medical decision making (COMRADE satisfaction with communication: Ft36=0.642, P=.43; COMRADE confidence in decision: F^36=2.310, P=.14). Patients did, however, differ on the secondary outcome self-reported satisfaction with care (Fb37=6.306, P=.02). Patients who received the allocated intervention were less satisfied than patients who did not.

Table 3. Primary outcome data of patients’ perceived involvement in medical decision making at the end of the study using the Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE) test.

COMRADE subscalea

Intervention, mean (SD)

Control, mean (SD)

F (df)

P

Satisfaction with communication (n=73)

38.25 (1.06

37.19 (1.165)

0.422 (1,68)

.52

Confidence in decision (n=70)

38.78 (1.17)

38.72 (1.307)

0.086 (1,67)

.77

aGroup differences were analyzed using a general linear model with age and partner status as covariates.

Use of and Satisfaction With the Web-Based Decision Aid

Of the 48 patients who used the Web-based decision aid, 12 used their own computer, 12 used the computer at the clinic, and 6 used a computer elsewhere. Furthermore, 13 used the decision aid independently, 16 received assistance from a professional (often their case manager), and 1 received assistance from someone else. First-episode patients used their own computer and used the decision aid without assistance more often than chronic patients did. Of the 48 patients who used the website, 34 (71%) used full functionality of the Web-based decision aid, meaning that patients completed the care needs assessment (first webpage of the website) and looked through the digital catalog with descriptions of treatment modules (second webpage of the website). More than half of them were long-term care patients (27/48, 56%).

In the intervention condition, 29 of 48 patients who used the decision aid (60%) completed questions about their satisfaction with the decision aid. They agreed or completely agreed with the following statements: “I have been well informed about the treatment options offered by Friesland Mental Health Care Service by the decision aid” (22/29, 76%), “The advice presented by the decision aid has helped me to reflect on what I want” (22/29, 76%), “The decision aid was easy to use” (20/28, 71%), “I would recommend the decision aid to others” (20/27, 74%) and “The decision aid helped me to get a clearer view on what my problem areas or points of interest are” (17/28, 61%). Patients were divided on whether the decision aid helped them to better prepare the evaluation meeting with their clinicians, 44% (12/27) said it did help; 56% (15/27) were neutral or said it did not help. Means and standard deviations can be found in Table 4.

Table 4. Secondary outcome data of patients' satisfaction with the Web-based decision aid.

Question

Mean (SD)a

I have been well informed about the treatment options offered by the GGZ Friesland by the decision aid (n=29)

3.93 (0.84)

The advice presented by the decision aid has helped me to reflect on what I want (n=29)

3.86 (0.79)

As a consequence of using the decision aid, I was better prepared for the evaluation meeting with my clinician (n=27)

3.33 (0.78)

The decision aid helped me to get a clearer view on what my problem areas or points of interest are (n=28)

3.61 (0.92)

The decision aid was easy to use (n=28)

3.79 (1.07)

I would recommend the decision aid to others (n=27)

3.89 (0.75)

aScores ranged from 0 (completely disagree) to 5 (completely agree).

Discussion

Principal Findings

In this study, we report on a clinical trial and process evaluation of a Web-based intervention to facilitate shared decision making for people with psychotic disorders.

To be able to explore potential implementation issues and contextual problems within the trial, we conducted a process evaluation. This evaluation showed that no significant problems could be observed in the attitude and beliefs of clinicians. Participating clinicians had an overall positive attitude toward shared decision making. They reported that their patients were generally interested in and capable of participating in medical decision making, they considered patient decision aids be to potentially helpful, and they judged relatively few factors to be hampering in a shared decision-making process. However, problems were observed in the implementation of the intervention. Not all patients in the intervention group were actually offered the possibility to use the decision aid and, more importantly, ROM and treatment evaluation meetings in which the treatment plan was to be discussed in a process of shared decision making did not always take place. Moreover, interviews indicate that the Web-based intervention might have been framed differently to different patients, which may have shaped their expectations and affected their evaluation. An interesting finding in the process evaluation was that patients who perceived their involvement in medical decision making as low were judged by clinicians to be less capable of participating in decision making. This could imply that patients participate less because they are less capable. Nevertheless, we cannot rule out that patients participate less because clinicians consider them less capable and, therefore, provide less opportunities for patients to participate in decision making.

The findings of our trial show that more than one-third of the patients who were provided access to the Web-based decision aid chose to use it and most used full functionality of the decision aid whether they were first-episode patients or long-term patients. Users and nonusers did not differ in demographic variables. At least one-quarter of the patients used their own computer and a similar proportion used the decision aid without assistance. Most of these were first-episode patients. On average, users of the decision aid reported to be rather satisfied with the system. Nevertheless, primary outcome results could not support the assumption that the use of electronic decision aids increases patient involvement in medical decision making, neither in intention-to-treat analyses nor in per protocol analyses. In addition, we did not find a difference in self-reported satisfaction with care between patients who had the opportunity to use the decision aid versus those who did not.

Our outcomes are in-line with the study by Woltmann et al [12] who found no difference in patient satisfaction between intervention and control group. However, they contradict the findings by Hamann et al [3] and Steinwachs et al [13] who found a positive effect of decision aids on patients' involvement in consultations with their clinicians. This discrepancy can be explained by several reasons. First, the decision aids used in these trials differed in format (Hamann et al [3] used a printed decision aid) and content. Some decision aids primarily concentrated on pharmacological information, whereas others had a broader focus. Second, settings were different. In our study, patients could use the decision aid either in the clinic or at home, with or without assistance, whereas in the trial by Hamann et al [3], patients used the decision aid in a psychiatric ward with assistance of trained nurses. The setting in the study by Steinwachs et al [13] was not described. Third, our response rate was very low. This is partly because of the naturalistic setting of our study. However, response rates are highly dependent on selection criteria used in studies. For example, if

Steinwachs et al [13] included all eligible patients (eg, not excluding patients who were considered unsuitable by their clinician), their response rate would have been comparable. Fourth, the outcome measures used in our study might have been too unspecific, indirect, or insensitive to detect differences in a small sample. The COMRADE measures patients’ perceived involvement in medical decision making with a self-report questionnaire that is completed retrospectively. What actually happens during the conversation between patient and clinician remains a black box. Furthermore, research has shown that ratings on patient satisfaction questionnaires tend to be more optimistic than patients’ actual evaluations [28,29], implying that there may be less differentiation in the response behavior. Finally, discrepancies could, but are not likely to, be explained by lack of need for shared decision making in our patient sample. Patients’ mean score on the API, which indicates their preference for participation in medical decision making, was comparable to or even slightly higher than previous studies in people with schizophrenia [2,3,30].

Strengths and Limitations

Given the problems observed in the process evaluation, the intervention designed for our study appeared not to fit in optimally with the routine practice of the participating clinical care teams. Therefore, the lack of significant effects on our outcome measures cannot be solely attributed to failures intrinsic to the intervention. Future studies might benefit from a stronger integration of shared decision-making interventions in clinical practice by training clinical teams in using (output) from decision aids. A comprehensive overview of the working flow of patients and clinicians is crucial to realize this integration. Given the low response rate and moderate participation rate in this study, it may also be desirable to investigate efficacy of decision aids in a less naturalistic setting in which participating patients are selected more strictly and required to use the decision aid before performing a naturalistic study. In addition, special attention should be paid to the selection of outcome measures used to assess the shared decision-making process. Instruments focusing on satisfaction might suffer from ceiling effects, and instruments such as the COMRADE may be too broad and indirect to detect changes in the decision-making process. A better alternative is to record conversations between van der Krieke et al clinicians and patients and observe what is actually happening within that conversation. A promising instrument for this may be the recently developed Mappin’SDM [31], which combines patient, clinician, and observer perspectives. It is also important to note that using Web-based decision aids or support systems does not need to be a desirable target for all patients. Although some may benefit from new tools, others might not. It would be most helpful to know what works for whom.

The main limitation of this study is the weak implementation of the study protocol; as a result, it is difficult to draw firm conclusions about the study’s outcomes. We tried to prevent this by preparing the participating teams before the start of the trial and keeping closely in touch during the trial (eg, being present at clinical meetings, functioning as helpdesk, sending individual emails to participating clinicians as reminders of specific actions). Another important limitation is the large numbers of dropouts before the follow-up measurement, even though patients were offered a small gift for returning their completed questionnaire.

Our study also has strengths. Most importantly, it affirms previous findings that many people with a severe mental illness can work with electronic decision aids, either with or without assistance, at the clinic or at home. Furthermore, our study provides insight in variation among the population concerning interest in and use of electronic decision aids. Our results suggest that part of the population is not able or does not feel the need to work with these decision aids. Based on our results, the ratio of users versus nonusers could be 50-50. Another strength is that we collected detailed information about allocation and reception of the intervention with varying illness durations, and we included a process evaluation that allowed us to perform a critical analysis on the trial results.

Conclusion

The development of electronic decision aids to facilitate shared medical decision making is encouraged and many people with a psychotic disorder can work with them. This holds for both first-episode patients and long-term care patients, although the latter group might need more assistance. However, effects of decision aids on patient participation in medical decision making have not been consistently demonstrated.

Acknowledgments

This study was supported by the Netherlands Organisation for Health Research and Development (ZonMw); Fonds Psychische Gezondheid; ICT regie; and the Dutch Ministry of Health, Welfare and Sport under the name WEGWEIS (grant number 300020011).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Video of the Web-based decision aid.

[AVI File, 125MB - jmir_v15i10e216_app1.avi ]

Multimedia Appendix 2

CONSORT-EHEALTH Checklist V1.6.2 [32].

[PDF File (Adobe PDF File), 993KB - jmir_v15i10e216_app2.pdf ]

References

Abbreviations

API: Autonomy Preference Index

CANSAS-P: Camberwell Assessment of Need Short Appraisal Schedule-Self-Report Version

COMRADE: Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness CSQ: Client Satisfaction Questionnaire

DSM-IV-TR: Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Text Revision)

GAF: Global Assessment of Functioning

HoNOS: Health of the Nation Outcome Scales

MANSA: Manchester Short Assessment of Quality of Life

METiGG: Medisch-ethische Toetsingscommissie instellingen Geestelijke Gezondheidszorg (Dutch medical ethical committee for mental health care)

NTR: NederlandsTrial Register (Dutch trial register)

PANSS: Positive and Negative Syndrome Scale

ROM: Routine Outcome Monitoring

Edited by G Eysenbach; submitted 26.07.13; peer-reviewed by VShaffer, T Thyvalikakath; comments to author 16.08.13; revised version received29.08.13; accepted 15.09.13; published 07.10.13

Please cite as:

van derKrieke L, Emerencia AC, Boonstra N, Wunderink L, de Jonge P, Sytema S

A Web-Based Tool to Support Shared Decision Making for People With a Psychotic Disorder: Randomized Controlled Trial and Process Evaluation

J Med Internet Res 2013;15(10):e216

URL: http://www.jmir.org/2013/10/e216/

doi: 10.2196/jmir.2851

PMID:24100091

©Lian van der Krieke, Ando C Emerencia, Nynke Boonstra, Lex Wunderink, Peter de Jonge, Sjoerd Sytema. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.10.2013. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.


Contents lists available at ScienceDirect

Schizophrenia Research: Cognition

journal homepage: http://www.schizrescognition.com/

e-Motional Training®: Pilot study on a novel online training program on social cognition for patients with schizophrenia

Miriam Vazquez-Campo a,b, Yolanda Marono c, Guillermo Lahera d, Raimundo Mateos a, Alejandro Garcia-Caballero a,b,36

a Department of Psychiatry, School of Medicine, University of Santiago de Compostela, Santiago de Compostela, Spain

b University Hospital Complex of Ourense, Ourense, Spain

c Department of Clinical Psychology and Psychobiology, Faculty of Psychology, University of Santiago de Compostela, Santiago de Compostela, Spain

d Department of Medicine and Medical Specialties, University of Alcala IRyCIS, CIBERSAM, Madrid, Spain



ARTICLE INFO

ABSTRACT

Article history:

Received 18 June 2015

Received in revised form 20 November 2015

Accepted 25 November 2015

Available online 12 March 2016

Background: Patients with schizophrenia have deficits in social cognition (SC), a construct that includes emotion perception (EP), social perception (SP), theory of mind (ToM) and attributive style (AS). The aim of our study was to assess the applicability, identify potential problems and obtain preliminary data on the efficacy of a new online training program on SC called e-Motional Training (ET®), which can be remotely administered and remotely supervised by a clinician.

Keywords:

Social cognition

Schizophrenia

Computerized training Cognitive remediation Theory of mind

Materials and methods: A pre/post intervention pilot study was conducted with 21 patients with schizophrenia in the healthcare area of Ourense, Spain (12 patients were assigned to the intervention group and 9 in the control group). The control group received standard treatment (TAU) (occupational therapy and leisure group). The intervention group received TAU plus 12 sessions (1 hour per week) with ET® (including training modules on emotional perception and a short animated cartoon for ToM and AS training, including automated metacognitive feedback).

Endpoints: EP (Ekman 60 Faces Test), ToM (Hinting Task, Faux Pas, Happe), AS (Ambiguous Intentions Hostility Questionnaire).

Results: ET® was shown to be an understandable, viable and pleasant program for the participants. After the intervention, statistically significant data (p < 0.05) were obtained for the EP, ToM and AS variables.

Conclusion: ET® enables self-training in SC and online follow-up by the therapist, thereby covering the lack of online intervention instruments validated for patients with SC deficits. Our preliminary results demonstrate the feasibility of ET® and its possible efficacy in improving emotion recognition, ToM and AS.

© 2016 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Background

Social cognition (SC) is a psychological construct that refers to a collection of mental operations that underlie social interactions. SC includes the processes involved in the perception, interpretation and generation of responses when faced with the intentions, dispositions and behavior of others (Green et al., 2008; Penn, Sanna, and Roberts, 2008).

SC includes emotion perception, both in peoples faces and voices (Kohler et al., 2010; Tseng et al., 2013); social perception (Couture et al., 2006; Pinkham, 2014) i.e., the interpretation of clues about what happens in a certain social context and the application of this social understanding to develop more appropriate behavior; theory of mind (ToM) (Brune, 2005b; Sarfati et al., 1999), which is defined as the

ability to be aware that others have different ideas and intentions than ours; and attributive style (AS), which refers to peoples tendency to explain events that have occurred in their life and that sometimes leads them to consider the negative events as caused by the misconduct of others (Green et al., 2008; Hasson-Ohayon et al., 2014; Langdon et al., 2013; Mizrahi et al., 2008). In recent years, this topic has been the subject of intense study, revealing disorders in emotion perception, ToM, AS and social perception in patients with schizophrenia (Chung et al., 2014; Kurtz and Richardson, 2012; Lahera et al., 2014; Savla et al., 2013). The study of these deficits and of strategies for improving them is important because SC appears to have a greater repercussion on social function than neurocognition itself (Bigelow et al., 2006; Brune, 2005a; Green et al., 2008) and is considered to be a mediator between neurocognition and functional performance (Casacchia et al., 2004; Penn et al., 1996; Pinkham and Penn, 2006). SC rehabilitation has been made possible by various therapeutic models (Addington et al., 2006; Addington et al., 2010). These rehabilitation models initially focused on the constructs specific subdomains (Combs et al., 2009; Frommann et al., 2003;

http://dx.doi.org/10.1016/j.scog.2015.11.007

2215-0013/© 2016 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).


Kayser et al., 2006; Penn and Combs, 2000; Roberts and Penn, 2009; Roder, Mueller, and Schmidt, 2011; Russell et al., 2006; Silver et al., 2004). More ambitious models subsequently emerged that included all components of SC (Horan et al., 2009; Roberts et al., 2014; Rocha and Queiros, 2013). However, all of these models required a significant number of sessions (between 12 and 45), were typically applied in a group setting and required specific training by the therapists, factors that hindered their application in the clinical setting (Roberts et al., 2010). With this objective in mind, we designed an online rehabilitation program for SC, limited to just 12 sessions (the minimum number of face-to-face sessions reported in previous studies.)

Computerized training, conversely, allows for their application with minimal supervision, which enables access by the entire target population and produces clear financial savings (Ventura etal., 2013).

Computer-based programs can be classified into three groups:

However, apart from improvements in emotion recognition, to date none of these strategies has demonstrated robust improvements in ToM or AS, and with the exception of Social Ville none is intended for self-training.

e-Motional Training® (ET®) allows online self-training and stores the data of each individual session. ET® is designed following the basic principles of neuropsychological rehabilitation in this domain (Brenner et al., 1987; Ochsner, 2008; Zubin and Spring, 1977). The program aims to deliver realistic and natural but attractive exercises of short duration without irrelevant stimuli or distractions, while offering continuous feedback. Emotion recognition tasks are designed with increasing difficulty, starting with tutorials, following with eyes and mouths recognition and finally scaling to microexpression training. An animated short film with 33 scenes is the vehicle for ToM, social perception and attributional style stories. After each scene, a series of questions including ToM, AS and control questions are posed. When the answer is incorrect, the patient receives metacog-nitive suggestions, which lead the user to think about the situation from a different perspective or prompts the user to pay attention to specific aspects of the film.

The program was composed of 12 1-hour sessions (the minimum number of face-to-face sessions reported in previous studies).

In this study, we present the feasibility and applicability results of our SC rehabilitation program: e-Motional Training® (ET®).

1.1. Objective

To assess the applicability and to explore the efficacy of a new online training program on SC in patients with chronic schizophrenia: e-Motional Training®.

Based on the characteristics of our program and the previous literature, we hypothesized that ET® would improve the three assessed SC domains (emotional recognition, ToM, EA) in the patients with schizophrenia but would not change the patients' symptoms (PANSS).

2. Methodology

A pilot, pre/post intervention study on patients with chronic schizophrenia was performed. The patients were recruited at 3 centers: The University Hospital of Ourense (CHUO), the Association of Persons and Families with Mental Illness of Ourense (MOREA) and the Cebolino Community Residence (Ourense).

Inclusion criteria: We included patients who voluntarily agreed to participate in the study, aged 18-50 years with a diagnosis of schizophrenia (DSM-IV TR), who were clinically stable (no acute psychotic symptoms and not hospitalized during the last three months), and who had no comorbidity with other psychiatric or neurological diseases (International Neuropsychiatric Interview-MINI) and excluding current substance abuse (except nicotine).

2.1. Description of the intervention

The control group received standard treatment (TAU) (occupational therapy and leisure group).

The intervention group received TAU plus 12 sessions (1 hour per week) with ET® (including training modules on emotional perception and a short animated cartoon for ToM and AS training, including automatic metacognitive feedback). Some 30% of the participants participated in the intervention from their homes, visiting the center only for the assessments and for the first session of the intervention. The rest of the participants (70%) performed the intervention at the healthcare center under direct supervision. All participants in the intervention group (regardless of mode: clinical or online) completed the same number of sessions.

To start the intervention, the patient accessed the website www. e-motionaltraining.com and registered with a username and password.

The first 4 meetings (1 h each session) are dedicated to recognizing facial emotions. This section includes a pre and post test, tutorials, and scaling minigames starting with eyes and mouths and finally microexpression (<250 ms) training (Fig. 1).

The next 8 sessions (1 h each) include watching a short, interactive animated cartoon in which a couple invites their friends to their home for a party. As the story unfolds, instances of miscommunication occur among the actors, causing various emotions and mental conditions such as anger, affection, appreciation and jealousy. After each scene, the user is queried about what happened, with questions about ToM (interpreting irony, insinuations, faux pas, 2nd-order false beliefs, etc.), social perception (interpretation and analysis of the social situation through the visual content of each scene) and attributive style (the individuals' attributions to the events, and questions such as, What kind of thinking would result in Cristina getting better results in this situation?), as well as control questions. The game provides user feedback and, in the event of errors, can display a hyperlink with information and metacognitive strategies, whose objective is to help users understand the scene that they just watched (Fig. 2).

In our study, the researcher (MVC) monitored the patients' performance and resolved the initial questions regarding computer and software use, but did not provide any assistance with the training tasks.


Apart from offering immediate feedback and a progress summary, e-Motional Training® stores the results of each session individually in a database with access restricted to the therapist or researcher.

2.2. Instruments

The post training assessment was performed during the month following the end of treatment at patients' convenience, using the following battery of tests37:

MVC was responsible for pre and post testing.

3. Statistical analysis

A descriptive analysis was first performed. The qualitative variables are presented with their absolute frequency and percentage. The quantitative Gaussian variables are presented as mean and standard deviation, and the non-Gaussian variable is presented as median [minimum-maximum]. To compare 2 qualitative variables we used the chi-squared test.

To compare the quantitative variables, we used Student's t-test for independent samples and for paired samples (in Gaussian variables) and U-Mann Whitney, Wilcoxon (in non-Gaussian). The accepted a risk was 0.05. The data were analyzed using SPSS 15.0 software.

4. Results

A total of 31 participants were recruited during a 6-month period, 22 of whom met the selection criteria. Twelve participants were included in the intervention group, and 9 were included in the control group. There were two patients lost to follow-up in the intervention group (both occurred before the ET training), one was a relapse and the other committed suicide (Fig. 3).

All patients provided informed consent (both verbal and written) to participate in this study.

All variables followed a normal distribution, except for happiness in Ekman's 60 faces, due to ceiling effects.

There were no significant differences in the demographic and clinical characteristics between the 2 groups. At baseline, there were statistically significant differences in the total results from Faux Pas (p = 0.044), thus the control group demonstrated poorer performance.

At the start of the study, the participants were a mean age of 39.47 (±9.10) years. Table 1 shows the characteristics and demographic variables of the 2 groups.


TRAINING PROGRESS


TOTAL


Fig. 2. Training module on ToM and AS.

The enrolled patients were diagnosed with schizophrenia at the age of approximately 24 (±8.42) years, were taking a dosage equivalent to 424 mg of chlorpromazine per day and had been hospitalized once due to exacerbations of their psychiatric disease (Table 1).

4.1. Feasibility

Except for the 2 patients lost to follow-up before starting the intervention, all participants in the intervention group completed the training at the scheduled meetings. Ninety percent of the participants stated that they found the game to be easy, and all declared that ET was entertaining and amusing, suggesting their interest in performing similar computer activities in the future. Some 70% found the program useful for improving their social relations and 30% found it very useful. Regarding online self-treatment, 57% of patients trained at the health facility stated their willingness to play it at home. Some 90.9% of the patients expressed their interest in participating in similar online interventions.

4.2. Emotion recognition

At baseline, both groups had difficulties with fear, sadness and anger whereas normal results were obtained for happiness and surprise (Table 2). One month after the intervention, the intervention group showed a statistically significant improvement in total perception scores, with greater improvements in the emotions more affected at baseline. The control group showed a significant decline in total perception scoring.

4.3. ToM

Baseline scores on the Faux Pas test were under the standard cutoff point of 54 out of 60 points. After training, the intervention group showed a significant improvement in Faux Pas detection question scores (p < 0.001)

Baseline scores on Happes Strange Stories were under the standard cutoff point of 14 out of 16. After training, the intervention group showed a statistically significant improvement (p = 0.01), and there were no changes were in the control group.

Baseline scores on the Hinting Task test revealed a deficit in this area, with a score of 14.56 for the control group and 12.70 for the intervention group, respectively (maximum score is 20 points). Patients in the intervention group showed a significant improvement in comparison with the control group (p = 0.006).

Regarding attributive style, we used the Ambiguous Intent Attribution Questionnaire (AIHQ), which explores the attributional cognitive biases (hostility, intent, guilt, anger and aggressiveness) in 3 types of situations: ambiguous (AIHQ-AM), intentional (AIHQ-INT) and accidental (AIHQ-AC). Both groups showed improved performance in the total AIHQ scores after completing the study (Table 3). These differences were more pronounced in the intervention group, with asignificant improvement in the answers for ambiguous (p = 0.002) and intentional (p = 0.031) situations.

The patients included in the study (control and intervention groups) showed responses at baseline within the normal range in emotional intelligence as measured with the MSCEIT. There were no improvements in MSCEIT scores after the intervention or TAU.

No differences were found between the participants in the intervention group who performed the intervention from their homes and those who were directly supervised at healthcare facilities.

Finally, there were statistically significant improvements in the PANSS scores in the intervention group (Table 3).

5. Discussion

The results of this pilot study suggest that ET® is an attractive intervention, with a wide age range, and it is feasible and reliable for use, even if the user has no previous experience with information technology.

Following the intervention with ET®, the difficulties in recognizing facial emotions were reduced, with patients achieving scores within the normal range and similar to those of the general population


(Dodich et al., 2014; Young et al., 1997). These results are consistent with those achieved by similar strategies for rehabilitating emotional perception, such as the strategy performed with METT (Russell et al., 2008) or NPT-SM (Fernandez-Gonzalo et al., 2015). After a single training session for identifying microexpressions, these researchers found an improvement in emotional recognition, results that are similar to those of a study performed with Emotion Management

Therapy (EMT) (Hodel et al., 1998). A pilot study was conducted with Training of Affect Recognition (TAR) (Frommann et al., 2003) a program that includes training on differentiating the 6 basic emotions, as well as training for integrating facial expressions into overall processing. The study results were not statistically significant, but subsequent results of clinical trials with TAR showed improvements in terms of emotion recognition skills (Wo et al., 2005) and a certain

Table 1

Participant characteristics.

Control Group

Intervention Group

p

Mean age years (SD)

41.78 (9.39)

37.40 (8.79)

0.31

Sex

Male, n

5

7

0.51

Female, n

4

3

Education level

Primary, n

4

3

0.38

Secondary, n

4

7

Tertiary, n

1

0

Occupation

Unemployed, n

1

2

0.46

Retired, n

8

6

Employed, n

0

1

Student, n

0

1

Marital status

Single, n

8

9

0.93

Married, n

0

0

Separated, n

1

1

Widowed/Other, n

0

0

Computer skills

None, n

2

3

0.46

User, n

6

4

Expert, n

1

3

Mean age at diagnosis (SD)

24.56 (10)

23.40 (7.24)

0.78

Mean treatment, mg of chlorpromazine (SD)

380.56 (101.37)

463.80 (105.92)

0.10

Hospital admissions, n (SD)

0.78 (1.09)

1.1 (0.99)

0.51

WAIS-Total

68.68 (10.95)

77.50 (13.89)

0.18

WAIS-VC

80.33 (18.46)

86.30 (12.79)

0.40

WAIS-PR

71.44 (17.40)

76.10 (16.51)

0.44

WAIS-WM

74.67 (11.03)

83.10 (14.07)

0.09

WAIS-PS

73.33 (5.87)

82.60 (13.42)

0.09

VC: Verbal Comprehension. PR: Perceptual Reasoning. WM: Working Memory. PS: Processing Speed.


Table 2

Scores pre and post intervention.

Control Group

Intervention Group

Pre Mean (SD)

Post Mean (SD)

p

Pre Mean (SD)

Post Mean (SD)

p

Ekman Total

38.44 (7.40)

35.78 (7.53)

0 .025’’

44 (4.47)

55.60 (2.17)

<0 .001’’

Happiness +

10[6-10]

9 .92[6-10]

0 .999

10[9-10]

10[9-10]

0 .317

Surprise

7.56 (2.60)

7 (2.92)

0 .139

8.30 (1.16)

9.70 (0.48)

0 .007’’

Fear

4.22 (2.68)

3.44 (2.70)

0 .133

5.90 (2.33)

8.60 (1.77)

0 .002’’

Sadness

5.89 (2.03)

5.11 (2.26)

0 .133

5.30 (1.56)

9 (0.94)

0 .001’’

Disgust

6 (2.92)

6 (2.29)

0 .999

7.80 (2.15)

9.80 (0.42)

0 .010’’

Anger

5 (2.69)

4.11 (2.42)

0 .069

5.90 (1.52)

1.16

0 .001’’

p-value: Paired samples t-test. +: Median [min-max] and p-value: Wilcoxon test. ’’ p < 0.05 = statistical significance.


generalization on other measures of social cognition, including ToM (Wolwer and Frommann, 2011).

After training with ET®, the patients with chronic schizophrenia had a statistically significant improvement in emotional perception, detection of gaffes (Faux Pas) and other ToM scales (Hinting Task and Happe tests). However, their final scores did not reach the performance ranges of the general population. A number of studies conducted with the Social Cognitionand InteractionTraining (SCIT) group intervention (Lahera et al., 2013; Penn et al., 2005) and with Metacognitive and Social Cognition Training (Rocha and Queiros, 2013) have also demonstrated improved emotional perception and ToM. However, with regard to the latter, other studies performed with SCIT had conflicting results (Roberts et al., 2014).

We found differences in our pilot study regarding attributive style between the groups. The vast majority of published studies using SCIT (Lahera et al., 2013; Penn et al., 2005), Emotion and ToM Imitation Training (Mazza et al., 2010), Social Cognitive Skills Training (SCST) (Horan et al., 2009) and Metacognitive and Social Cognition Training (Rocha and Queiros, 2013), found no differences in this variable. Only one study, using SCIT (Combs et al., 2007a), showed a reduction in hostile attributions toward others, with a medium to large effect size.

Our study found no differences regarding emotional intelligence (as assessed with MSCEIT) following the intervention; this was also the case in other studies (Lindenmayer et al., 2012; Nahum et al., 2014; Rocha and Queiros, 2013). Additionally, the patients had baseline scores close to or within the normal range, which contradicts the data from the other SC scales and their own clinical impression. These facts prompt us to doubt on the validity of MSCEIT for the assessment of schizophrenia patients.

Finally, our results showed a significant improvement in PANSS scores, which can be attributed to an amelioration of negative symptoms. It can be argued that PANSS improvement is on the basis of the SC reported changes, but despite its occurrence, this association was not reported in previous studies (Kayser et al., 2006; Rus-Calafell et al., 2014), or in a meta-analysis of 19 clinical trials that assessed the efficacy of various SC interventions (Kurtz and Richardson, 2012).

ET® was feasible and well accepted, allowing individual selftraining and online follow-up, favoring accessibility and autonomy. Moreover, we have shown improvements in all the subdomains of SC except for social perception (which was not assessed).

5.1. Limitations of the study

The study was not randomized. The patients were assigned to the various treatments as convenient in a consecutive manner. The first 12 patients were assigned to the intervention group and the remainder to the control group. Due to the study's exploratory nature, we did not add assessments that measured generalization, using scales that assessed functionality, but perhaps this fact might have provided relevant information. The patient assessments were not blinded, and a number of these tests have no objective scoring methods.

Contributors

MVC, AAGC and YMS devised and designed the study and acquired and interpreted the data. They also drafted the article.

GLF and RMA performed the critical review of the intellectual content and approved the final version of the submitted article.

Table 3

Scores pre and post intervention.

Control Group

Intervention Group

Pre Mean (SD)

Post Mean (SD)

p

Pre Mean (SD)

Post Mean (SD)

p

Happe

6.67 (4.27)

6.89 (4.31)

0.512

8.20 (3.58)

11.20 (4.68)

0.010’’

Faux Pas

18.78 (12.89)

20.56 (15.33)

0.353

31.40 (12.43)

38.30 (19.20)

0.131

Detect. FP

3.67 (2.39)

3.67 (2.59)

0.999

5.90 (2.37)

8 (2.98)

< 0.001’’

Hinting

14.56 (3.28)

14.56 (3.94)

0.999

12.70 (4.45)

16.90 (4.82)

0.006’’

AIAQ-T

172.56 (21.11)

166.22 (20.56)

0.076

178.40 (54.98)

167.40 (51.01)

0.131

AIAQ-AM

62.56 (7.76)

62.89 (7.97)

0.814

69.20 (25.34)

61 (22.70)

0.002’’

AIAQ-INT

65.78 (10.47)

65 (8.60)

0.622

74.80 (19.60)

72.70 (21.25)

0.031’’

AIAQ-AC

44.22 (10.95)

40.44 (12.46)

0.062

44.50 (15.75)

41.30 (13.50)

0.064

MSCEIT

86.56 (13.50)

82.78 (9.06)

0.124

96.10 (14.36)

95.60 (10.45)

0.763

PANSS-T

26.11 (6.27)

26 (6.44)

0.729

36.40 (14.54)

29.80 (9.88)

0.012’’

PANSS-P

11.89 (4.34)

11.78 (4.23)

0.347

17.30 (7.18)

16.40 (7.60)

0.753

PANSS-N

14.22 (3.66)

14.22 (3.76)

0.999

19.80 (8.67)

14.40 (6.72)

0.098

* p-value: Paired samples t-test. ’’ p < 0.05 = statistical significance.


Conflict of interest

The authors declare no conflicts of interest.

Acknowledgments

This study was made possible with the help of the College of Physicians of Ourense, by granting Cabaleiro Goas-Prize and the College ofPsychologists of Galicia for the award of the Siota grant.

References

Addington, J., Saeedi, H., Addington, D., 2006. Influence of social perception and social knowledge on cognitive and social functioning in early psychosis. Br. J. Psychiatry 189, 373-378. http://dx.doi.org/10.1192/bjp.bp.105.021022.

Addington, J., Girard, T.A., Christensen, B.K., Addington, D., 2010. Social cognition mediates illness-related and cognitive influences on social function in patients with schizophrenia-spectrum disorders.J. Psychiatry Neurosc. 35 (1), 49-54.

Aghotor, J., Pfueller, U., Moritz, S., Weisbrod, M., Roesch-Ely, D., 2010. Metacognitive training for patients with schizophrenia (MCT): feasibility and preliminary evidence for its efficacy. J. Behav. Ther. Exp. Psychiatry 41 (3), 207-211. http:// dx.doi.org/10.1016/j.jbtep.2010.01.004.

Baron-Cohen, S., 1997. Hey! It was just a joke! Understanding propositions and propositional attitudes by normally developing children and children with autism. Isr. J. Psychiatry Relat. Sci. 34 (3), 174-178.

Bigelow, N.O., Paradiso, S., Adolphs, R., Moser, D.J., Arndt, S., Heberlein, A., ... Andreasen, N.C., 2006. Perception of socially relevant stimuli in schizophrenia. Schizophr. Res. 83, 257-267. http://dx.doi.org/10.1016/j.schres.2005.12.856.

Brenner, H.D., Hodel, B., Kube, G., Roder, V., 1987. Cognitive therapy of schizophrenic patients: problem analysis and empirical results. Nervenarzt 58 (2), 72-83.

Brune, M., 2005a. Emotion recognition, theory of mind, and social behavior in schizophrenia. Psychiatry Res. 133 (2-3), 135-147. http://dx.doi.org/10.1016/j. psychres.2004.10.007.

Brune, M., 2005b. Theory of mindin schizophrenia: a review of the literature. Schizophr. Bull. 31 (1), 21-42. http://dx.doi.org/10.1093/schbul/sbi002.

Casacchia, M., Mazza, M., Roncone, R., 2004. Theory of mind, social development, and psychosis. Curr. Psychiatry Rep. 6 (3), 183-189.

Chung, Y.S., Barch, D., Strube, M., 2014. A meta-analysis of mentalizing impairments in adults with schizophrenia and autism spectrum disorder. Schizophr. Bull. 40 (3), 602-616. http://dx.doi.org/10.1093/schbul/sbt048.

Combs, D.R., Adams, S.D., Penn, D.L., Roberts, D., Tiegreen, J., Stem, P., 2007a. Social Cognition and Interaction Training (SCIT) for inpatients with schizophrenia spectrum disorders: preliminary findings. Schizophr. Res. 91,112-116. http://dx. doi.org/10.1016/j.schres.2006.12.010.

Combs, D.R., Penn, D.L., Wicher, M., Waldheter, E., 2007b. The Ambiguous Intentions Hostility Questionnaire (AIHQ): a new measure for evaluating hostile social-cognitive biases in paranoia. Cogn. Neuropsychiatry 12 (2), 128-143. http://dx.doi. org/10.1080/13546800600787854.

Combs, D.R., Penn, D.L., Tiegreen, J.A., Nelson, A., Ledet, S.N., Basso, M.R., Elerson, K., 2009. Stability and generalization of Social Cognition and Interaction Training (SCIT) for schizophrenia: six-month follow-up results. Schizophr. Res. 112 (1-3), 196-197. http://dx.doi.org/10.1016/j.schres.2009.04.010.

Corcoran, R., Mercer, G., Frith, C.D., 1995. Schizophrenia, symptomatology and social inference: investigating theory of mindin people with schizophrenia. Schizophr. Res. 17 (1), 5-13.

Couture, S.M., Penn, D.L., Roberts, D.L., 2006. The functional significance of social cognition in schizophrenia: a review. Schizophr. Bull. 32, 44-63. http://dx.doi.org/ 10.1093/schbul/sbl029.

Dodich, A., Cerami, C., Canessa, N., Crespi, C., Marcone, A., Arpone, M., ... Cappa, S.F.,

Ekman, P., Friesen, W., 1976. Pictures of facial affect. Consulting Psychologists Press, Palo Alto, CA.

Extremera, N., Fernandez-Berrocal, P., 2009. Adaptation espanola del test de inteligencia emocional de Mayer-Salovey-Caruso (MSCEIT): manual y cuadernillo (TEA, Ed.). Madrid.

Fernandez-Gonzalo, S., Turon, M., Jodar, M., Pousa, E., Hernandez Rambla, C., Garcia, R., Palao, D., 2015. A new computerized cognitive and social cognition training specifically designed for patients with schizophrenia/schizoaffective disorder in early stages of illness: a pilot study. Psychiatry Res. 228 (3), 501-509. http://dx.doi. org/10.1016/j.psychres.2015.06.007.

Frommann, N., Streit, M., Wolwer, W., 2003. Remediation of facial affect recognition impairments in patients with schizophrenia: a new training program. Psychiatry Res.117 (3),281-284.

Gil, D., 2012. Adaptation al espanol de la prueba de teoria de la mente Hinting Task. Rev. Psiquiatr. Salud Ment. (Barc) 5 (5), 79-88.

Gil Sanz, D., Diego Lorenzo, M., Bengochea Seco, R., Arrieta Rodriguez, M., Lastra Martinez, I., Sanchez Calleja, R., Alvarez Soltero, A., 2009. Efficacy of a social cognition training program for schizophrenic patients: a pilot study. Span. J. Psychol. 12 (1), 184-191.

Gil-Sanz, D., Fernandez-Modamio, M., Bengochea-Seco, R., Arrieta-Rodriguez, M., Perez-Fuentes, G., 2014. Efficacy of the Social Cognition Training Program in a sample of schizophrenic outpatients. Clin. Schizophr. Relat. Psychoses http://dx.doi. org/10.3371/CSRP.GIFE.013114.

Green, M.F., Penn, D.L., Bentall, R., Carpenter, W.T., Gaebel, W., Gur, C., ... Park, S., 2008. Social cognition in schizophrenia: an NIMH workshop on definitions, assessment, and research opportunities. Schizophr. Bull. 34 (6), 1211-1220. http://dx.doi.org/ 10.1093/schbul/sbm145.

Hasson-Ohayon, I., Mashiach-Eizenberg, M., Avidan, M., Roberts, D.L., Roe, D., 2014. Social cognition and interaction training: preliminary results of an RCT in a community setting in Israel. Psychiatr. Serv. 65 (4), 555-558. http://dx.doi.org/10. 1176/appi.ps.201300146.

Hodel, B., Brenner, H.D., Merlo, M.C., Teuber, J.F., 1998. Emotional management therapy in early psychosis. Br. J. Psychiatry Suppl. 172 (33), 128-133.

Horan, W.P., Kern, R.S., Shokat-fadai, K., Sergi, M.J., Wynn, J.K., Green, M.F., 2009. Social cognitive skills training in schizophrenia: an initial efficacy study of stabilized outpatients. Schizophr. Res. 107 (1), 47-54. http://dx.doi.org/10.1016/j.schres. 2008.09.006.

Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13 (2), 261-276.

Kayser, N., Sarfati, Y., Besche, C., Hardy-Bayle, M.-C., 2006. Elaboration of a rehabilitation method based on a pathogenetic hypothesis of theory of mind” impairment in schizophrenia. Neuropsychol. Rehabil. 16 (1), 83-95. http://dx.doi. org/10.1080/09602010443000236.

Kohler, C.G., Walker, J.B., Martin, E.A., Healey, K.M., Moberg, P.J., 2010. Facial emotion perception in schizophrenia: a meta-analytic review. Schizophr. Bull. 36 (5), 1009-1019. http://dx.doi.org/10.1093/schbul/sbn192.

Kurtz, M.M., Richardson, C.L., 2012. Social cognitive training for schizophrenia: a meta-analytic investigation of controlled research. Schizophr. Bull. 38 (5), 1092-1104. http://dx.doi.org/10.1093/schbul/sbr036.

Lahera, G., Benito, A., Montes, J.M., Fernandez-Liria, A., Olbert, C.M., Penn, D.L., 2013. Social cognition and interaction training (SCIT) for outpatients with bipolar disorder. J. Affect. Disord. 146 (1), 132-136. http://dx.doi.org/10.1016/j.jad.2012. 06.032.

Lahera, G., Herrera, S., Fernandez, C., Bardon, M., de los Angeles, V., Fernandez-Liria, A., 2014. Familiarity and face emotion recognition in patients with schizophrenia. Compr. Psychiatry 55 (1), 199-205. http://dx.doi.org/10.1016/j.comppsych.2013. 06.006.

Langdon, R., Still, M., Connors, M.H., Ward, P.B., Catts, S.V., 2013. Attributional biases, paranoia, and depression in early psychosis. Br. J. Clin. Psychol. 52 (4), 408-423. http://dx.doi.org/10.1111/bjc.12026.

Lindenmayer, J.-P., McGurk, S.R., Khan, A., Kaushik, S., Thanju, A., Hoffman, L., ... Herrmann, E., 2012. Improving social cognition in schizophrenia: a pilot intervention combining computerized social cognition training with cognitive remediation. Schizophr. Bull. 64, 1-11. http://dx.doi.org/10.1093/schbul/sbs120.

Mazza, M., Lucci, G., Pacitti, F., Pino, M.C., Mariano, M., Casacchia, M., Roncone, R., 2010. Could schizophrenic subjects improve their social cognition abilities only with observation and imitation of social situations? Neuropsychol. Rehabil. 20,675-703. http://dx.doi.org/10.1080/09602011.2010.486284.

Mizrahi, R., Addington, J., Remington, G., Kapur, S., 2008. Attribution style as a factor in psychosis and symptom resolution. Schizophr. Res. 104 (1-3), 220-227. http://dx. doi.org/10.1016/j.schres.2008.05.003.

Nahum, M., Fisher, M., Loewy, R., Poelke, G., Ventura, J., Nuechterlein, K.H., ... Vinogradov, S., 2014. A novel, online social cognitive training program for young adults with schizophrenia: a pilot study. Schizophr. Res. Cogn. 1 (1), e11-e19. http://dx.doi.org/10.1016/j.scog.2014.01.003.

Ochsner, K., 2008. The social-emotional processing stream: five core constructs and their translational potential for schizophrenia and beyond. Biol. Psychiatry 64 (1), 48-61.

Penn, D.L., Combs, D., 2000. Modification of affect perception deficits in schizophrenia. Schizophr. Res. 46 (2-3), 217-229.

Penn, D.L., Spaulding, W., Reed, D., Sullivan, M., 1996. The relationshipofsocialcognition to ward behavior in chronic schizophrenia. Schizophr. Res. 20 (3), 327-335.

Penn, D., Roberts, D.L., Munt, E.D., Silverstein, E., Jones, N., Sheitman, B., 2005. A pilot study of social cognition and interaction training (SCIT) for schizophrenia. Schizophr. Res. 80 (2-3), 357-359. http://dx.doi.org/10.1016Zj.schres.2005.07.011.

Penn, D.L., Sanna, L.J., Roberts, L., 2008. Social cognition in schizophrenia: an overview. Schizophr. Bull. 34 (3), 408-411. http://dx.doi.org/10.1093/schbul/sbn014.

Peyroux, E., Franck, N., 2014. RC2S: A Cognitive Remediation Program to Improve Social Cognition in Schizophrenia and Related Disorders. Frontiers in Human Neuroscience 8, 400. http://dx.doi.org/10.3389/fnhum.2014.00400.

Pinkham, A.E., 2014. Social cognition in schizophrenia. J. Clin. Psychiatry 75 (Suppl. 2), 14-19. http://dx.doi.org/10.4088/JCP.13065su1.04.

Pinkham, A.E., Penn, D.L., 2006. Neurocognitive and social cognitive predictors of interpersonal skill in schizophrenia. Psychiatry Res. 143 (2-3), 167-178. http://dx. doi.org/10.1016/j.psychres.2005.09.005.

Pousa, E., 1999. MeasurementofTheory ofMindin healthy adolescents: translation and cultural adaptation ofF. Happe's Theory ofMind Stories. Universidad Autonoma de Barcelona.

Roberts, D.L., Penn, D.L., 2009. Social cognition and interaction training (SCIT) for outpatients with schizophrenia: a preliminary study. Psychiatry Res. 166 (2-3), 141-147. http://dx.doi.org/10.1016/j.psychres.2008.02.007.

Roberts, D.L., Penn, D.L., Labate, D., Margolis, S.A., Sterne, A., 2010. Transportability and feasibility of Social Cognition And Interaction Training (SCIT) in community settings. Behav. Cogn. Psychother. 38, 35-47. http://dx.doi.org/10.1017/ S1352465809990464.

Roberts, D.L., Combs, D.R., Willoughby, M., Mintz, J., Gibson, C., Rupp, B., Penn, D.L.,

Rocha, N.B.F., Queiros, C., 2013. Metacognitive and social cognition training (MSCT) in schizophrenia: a preliminary efficacy study. Schizophr. Res. 150(1), 64-68. http:// dx.doi.org/10.1016/j.schres.2013.07.057.

Roder, V., Mueller, D.R., Schmidt, S.J., 2011. Effectiveness of integrated psychological therapy (IPT) for schizophrenia patients: a research update. Schizophr. Bull. 37 (Suppl. 2), S71-S79. http://dx.doi.org/10.1093/schbul/sbr072.

Rose, A., Vinogradov, S., Fisher, M., Green, M.F., Ventura, J., Hooker, C.,... Nahum, M.,

Rus-Calafell, M., Gutierrez-Maldonado, J., Ribas-Sabate, J., 2014. A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. J. Behav. Ther. Exp. Psychiatry 45 (1), 81-89. http://dx.doi.org/10.1016/ j.jbtep.2013.09.002.

Russell, T.A., Chu, E., Phillips, M.L., 2006. A pilot study to investigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool. Br. J. Clin. Psychol. 45 (Pt 4), 579-583. http://dx.doi.org/10.1348/ 014466505X90866.

Russell, T.A., Green, M.J., Simpson, I., Coltheart, M., 2008. Remediation of facial emotion perception in schizophrenia: concomitant changes in visual attention. Schizophr. Res. 103 (1-3), 248-256. http://dx.doi.org/10.1016/j.schres.2008.04.033.

Sarfati, Y., Hardy-Bayle, M.C., Brunet, E., Widlocher, D., 1999. Investigating theory of mind in schizophrenia: influence of verbalization in disorganized and non-disorganized patients. Schizophr. Res. 37 (2), 183-190.

Savla, G.N., Vella, L., Armstrong, C.C., Penn, D.L., Twamley, E.W., 2013. Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr. Bull. 39 (5), 979-992. http://dx.doi.org/10.1093/schbul/sbs080.

Silver, H., Goodman, C., Knoll, G., Isakov, V., 2004. Brief emotion training improves recognition of facial emotions in chronic schizophrenia. A pilot study. Psychiatry Res. 128 (2), 147-154. http://dx.doi.org/10.1016Zj.psychres.2004.06.002.

Tseng, H.-H., Chen, S.-H., Liu, C.-M., Howes, O., Huang, Y.-L., Hsieh, M.H.,... Hwu, H.-G., 2013. Facial and prosodic emotion recognition deficits associate with specific clusters of psychotic symptoms in schizophrenia. PLoS One 8 (6), e66571. http://dx. doi.org/10.1371/journal.pone.0066571.

Ventura, J., Wilson, S.A., Wood, R.C., Hellemann, G.S., 2013. Cognitive training at home in schizophrenia is feasible. Schizophr. Res. 143 (2-3), 397-398. http://dx.doi.org/ 10.1016/j.schres.2012.11.033.

WAIS-IV, 2012. Escala de inteligencia de Wechsler para adultos-IV. Pearson, Madrid.

Wo, W., Frommann, N., Halfmann, S., Piaszek, A., Gil-Sanz, D., Gaebel, W., 2005. Remediation of impairments in facial affect recognition in schizophrenia : efficacy and specificity of a new training program. Schizophr. Res. 80, 295-303. http://dx. doi.org/10.1016/j.schres.2005.07.018.

Wolwer, W., Frommann, N., 2011. Social-cognitive remediation in schizophrenia: generalization of effects of the Training of Affect Recognition (TAR). Schizophr. Bull. 37 (Suppl. 2), S63-S70. http://dx.doi.org/10.1093/schbul/sbr071.

Young, A.W., Rowland, D., Calder, A.J., Etcoff, N.L., Seth, A., Perrett, D.I., 1997. Facial expression megamix: tests of dimensional and category accounts of emotion recognition. Cognition 63 (3), 271-313.

Zubin, J., Spring, B., 1977. Vulnerability a new view of schizophrenia. J. Abnorm. Psychol. 86 (2),103-126.

AMERICAN PSYCHOLOGICAL

ASSOCIATION


Journal of Consulting and Clinical Psychology

© 2020 American Psychological Association

ISSN: 0022-006X


2020, Vol. 88, No. 10, 937-950

http://dx.doi.org/10.1037/ccp0000602


Internet-Based Self-Help for Psychosis: Findings From a Randomized Controlled Trial


Stefan Westermann

University Medical Center Hamberg-Eppendorf,

Hamburg, Germany


Nina Ruegg

University of Bern


This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


Thies Ludtke

UiT-The Arctic University of Norway


Steffen Moritz

University Medical Center Hamberg-Eppendorf, Hamburg, Germany


Thomas Berger

University of Bern

Objective: Cognitive-behavioral therapy (CBT) for psychosis is recommended in many national guidelines, but dissemination into routine health care remains poor. This study tests whether an 8-week, CBT-oriented, Internet-based intervention (IBI) for people with psychosis is feasible, effective, and safe compared to care as usual. Method: A sample of 101 people diagnosed with schizophrenia-spectrum disorders (age: M = 40.0, SD = 9.60; sex: 58% female) was randomly assigned to either an IBI for psychosis or a wait-list control condition. The primary outcome was a composite score of the positive factor of the Positive and Negative Syndrome Scale, the Launay Slade Hallucination Scale, and the paranoia checklist (clinicaltrials.gov, NCT02974400). Results: The composite score of psychotic symptom severity decreased more in the IBI condition than in the wait-list condition, reflected in the significant interaction of Time X Condition, F(1, 87.28) = 4.04, p = .047, dbetween = 0.24, 95% CI [—0.15, 0.63]. In the combined sample of participants who received immediate or delayed access to the intervention, the outcome improved further during the 6-month follow up period with a significant main effect of time, F(1, 69.35) = 9.59, p = .003, d = — 0.37, 95% CI [—0.66, —0.07]. Participants were satisfied with the intervention (89%), and many used the intervention as defined per protocol (52%; at least four completed modules). Adverse events were infrequent (4.9%). Conclusions: Internet-based, CBT-oriented interventions provide an add-on effect to care as usual and have the potential to narrow the psychological treatment gap in psychosis.


What is the public health significance of this article?

This study shows that people with psychotic symptoms such as hallucinations and delusions can benefit from cognitive- behavioral interventions that are delivered in a self-help modality via the Internet.


Keywords: schizophrenia, Internet-based intervention, self-help, cognitive-behavioral therapy

Supplemental materials: http://dx.doi.org/10.1037/ccp0000602.supp


This article was published Online First August 13, 2020.

© Stefan Westermann, Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; ©Nina Ruegg, Department of Clinical Psychology and Psychotherapy, University of Bern; © Thies Ludtke, Department of Psychology, UiT-The Arctic University of Norway; Steffen Moritz, Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf; © Thomas Berger, Department of Clinical Psychology and Psychotherapy, University of Bern.

Steffen Moritz and Thomas Berger contributed equally to the manuscript and share last (senior) authorship.

This study was funded jointly by the Swiss National Science Foundation (Project 159384) and the German Research Foundation (Project DFG Mo 969/17-1). The clinical trial is registered with clinicaltrials.gov (NCT02974400; date of registration: November 28, 2016). We thank J. Ahlf-Schumacher, F. Alt, F. Baumann, L. Blunier, L.-E. Braunschneider, L. Bucker, S. Christen, M. Denzer, L. Henkel, N. Kreienbuhl, A. Paulus, H. Platow, M. Rahmede, K. Rauprich, J. Ricciardi, M. Stojisavlevic, N. Werkle, and R. Zurcher for their help in carrying out this study and M. J. Muller for kindly providing data for a functional reference sample. Related aricles that come from the same dataset are Ludtke et al. (2020) and Moritz et al. (in press). The authors state that there are no conflicts of interest.

Correspondence concerning this article should be addressed to Stefan Westermann, Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Martinistrabe 52, 20255 Hamburg, Germany. E-mail: st.westermann@uke.de

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


Cognitive-behavioral therapy for psychosis (CBTp) is effective in research settings (e.g., Morrison, Turkington, Pyle, Spencer, & Hutton, 2014; Tarrier et al., 1998) and efficacious in clinical practice (e.g., Krakvik, Grawe, Hagen, & Stiles, 2013; Lincoln et al., 2012), with small to medium-sized effects compared to care as usual regarding positive symptoms such as delusions and auditory verbal hallucinations (Bighelli et al., 2018; but see Jauhar et al., 2014) but not negative symptoms (Velthorst et al., 2015). National guidelines recommend offering CBTp to patients diagnosed with schizophrenia spectrum disorders (e.g., Germany: Lincoln, Pedersen, Hahlweg, Wiedl, & Frantz, 2019; United Kingdom: National Institute for Health & Care Excellence, 2014). The dissemination of psychological interventions for positive symptoms in standard health care remains poor (Haddock et al., 2014), however, despite the enormous individual and societal burden of schizophrenia (Jin & Mosweu, 2017). Novel approaches for the dissemination of CBTp are needed, from improvements in training and supervision (e.g., Irfan, Muzaffar, Kingdon, Rathod, & Naeem, 2019) to alternative modalities for delivering interventions such as a self-help format (Hazell, Hayward, Cavanagh, Jones, & Strauss, 2018). In the domain of technology-enhanced health care (eHealth), mobilebased interventions (also called mHealth) and Internet-based interventions (IBI) are two common self-help modalities. Whereas mHealth approaches are implemented as a smartphone app or text messaging service and aim at supporting self-monitoring and selfmanagement (Barlow, Ellard, Hainsworth, Jones, & Fisher, 2005), IBIs are implemented through a website and focus on more comprehensive self-help that corresponds more closely on manuals of face-to-face therapy (Andersson, 2016).

Generally, psychological self-help can be successfully delivered as Internet-based intervention (IBI), according to a growing body of evidence (Andersson, 2016). Usually, IBIs are primarily selfguided, Internet-delivered self-help treatments with or without minimal therapeutic guidance (Andersson, 2016). Users work through texts (e.g., psychoeducation), view explanatory videos (e.g., illustration of a vicious cycle), and fill in interactive worksheets (e.g., mood protocols). The materials are organized in modules that correspond to chapters in self-help books. Working with IBIs does not usually require interaction with others, but a guide is often available who gives feedback or answers questions on demand using an internal e-mail system. Thus, an IBI is a technologically enhanced bibliotherapy; it is neither a communication tool for classical face-to-face therapy via the Internet nor a virtual self-help group meeting platform. In anxiety disorders and depression, for example, IBIs are effective (Andrews et al., 2018) and also contribute to minimizing the treatment gap because they are being implemented in general health care systems in a growing number of countries (Titov, Dear, Nielssen, Staples, & Kaldo,

Users diagnosed with schizophrenia who have neuropsychological deficits may especially benefit from the low-threshold Internet-based format as they can reread the self-help materials and work with the interactive worksheets as often as they want. People with psychosis who fear stigmatization might prefer the anonymous and self-determined approach of a self-help program. Therefore, Internet-based interventions are likely to mitigate stigma-related treatment barriers. In addition, severe suspiciousness and symptoms of common comorbidities of schizophrenia—such as social phobia or panic disorder (Kiran & Chaudhury, 2016)— might prevent patients from making use of outpatient psychological therapy. As research on Internet-based treatments for social anxiety disorder demonstrates (Stolz et al., 2018), IBIs have the potential to overcome such symptom-related treatment barriers. In sum, IBIs have great potential to narrow the treatment gap in psychosis, as they are already doing in depression and anxiety disorders. However, direct evidence of the efficacy, feasibility, and safety of IBIs for psychosis is scarce.

Self-help interventions for psychosis in general have a small to medium-sized effect of Cohen’s d = 0.33 on overall symptom severity, according to a meta-analysis conducted by Scott, Webb, and Rowse (2015). The small number of studies that directly investigated IBIs—that is, online self-help interventions—for psychosis have resulted in promising (Alvarez-Jimenez et al., 2014) but mixed findings. Moritz and colleagues (2016) reported a significant reduction in the primary outcome of depressive symptom severity in an IBI compared to wait-list in a sample of 58 patients with schizophrenia spectrum disorders; Gottlieb, Romeo, Penn, Mueser, and Chiko (2013) found a reduction in the overall severity of auditory hallucinations in 21 patients with schizophrenia spectrum disorders in a web-based intervention, using an uncontrolled prepost design; and Rotondi and colleagues (2010) demonstrated, in a sample of 31 persons with schizophrenia or schizoaffective disorder using an RCT design, that a telehealth intervention can reduce positive symptoms. In contrast, Ruegg, Moritz, and Wester-mann (2018) did not find any effect of an Internet-based version of the metacognitive training—which has proven effective in group settings (Moritz, Andreou, et al., 2014)—in 15 participants with psychosis in an uncontrolled prepost design. Taken together, due to the paucity of research and mixed findings, it has been unclear to date whether IBIs are effective and thus can contribute to reducing the treatment gap in schizophrenia.

Although IBIs for psychosis seem to be promising, negative effects and experiences such as adverse events (e.g., suicidality) and aversive experiences (e.g., feeling a lack of social support in the program) have not been investigated. The investigation of negative effects of IBIs has increased lately (Ebert, Donkin, Andersson, Andrews, & Cuijpers, 2016; Rozental, Boettcher, Andersson, Schmidt, & Carlbring, 2015). For example, in depression the frequency of symptom deterioration is lower among people in IBIs than in control groups, according to a recent individual participant data meta-analysis with 3,805 participants (Karyotaki, Kemmeren, Riper, Twisk, & Cuijpers, 2018). In line with that, previous pilot studies reported no adverse events of IBIs for schizophrenia (Gottlieb et al., 2013; Moritz et al., 2016; Ruegg et al., 2018), and a review on e-mental health self-management for psychosis (e.g., medication management) indicated no negative effects (van der Krieke, Wunderink, Emerencia, de Jonge, & Sytema, 2014). Still, research on subtler negative effects such as the integration of the

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


self-help platform into a delusional belief or aversive experience in self-help platforms such as experiences of abandonment or pa-tronization (Westermann, Cavelti, Heibach, & Caspar, 2015) is so far missing in the domain of IBIs for psychosis.

Lastly, the feasibility and acceptance of complex IBIs for psychosis require further study. The majority of IBIs are accessible via a website using a browser (e.g., Ludtke et al., 2018) or via an app for smartphones (e.g., Stolz et al., 2018). Within a typical self-help platform, users work through several modules that address specific treatment targets such as behavioral activation and consist of texts, illustrations, interactive worksheets (e.g., ABC protocols), and psycho-educative videos. Thus, IBIs impose high cognitive demands on users (Rotondi, Eack, Hanusa, Spring, & Haas, 2015). In addition, IBIs usually offer a high degree of independent work and no, or only a low level of, personal support, which, if offered, consists of written guidance by a moderator who provides feedback, answers open questions, and sends reminders (Andersson, 2016). Thus, one could argue that the cognitive load and degree of autonomous work that a regular IBI requires is too demanding for users with schizophrenia, which means that the intervention and the user interface should, for example, be less complex (Rotondi et al., 2015). On the other hand, (a) studies highlight the importance of an engaging and appealing delivery format of IBIs for psychosis (Berry, Lobban, Emsley, & Bucci, 2016), (b) the severity of neurocognitive deficits in schizophrenia tends to be overemphasized (Moritz, Goritz, et al., 2017), (c) a meta-analysis of self-help in psychosis reported that interventions with higher complexity are more effective (Scott et al., 2015), and (d) many patients with schizophrenia are motivated to experience themselves as autonomous (Westermann et al., 2015), which requires space for decision-making and at least a minimum of program complexity. Thus, one could also argue that IBIs for schizophrenia should be complex and stimulating. Today, the majority of scientifically investigated IBIs for psychosis are on the left-hand side of the continuum from simplicity to complexity (e.g., Rotondi et al., 2010). Therefore, it is not possible currently to make bold claims regarding whether the benefits of a complex and demanding intervention outbalance the possible disadvantages, such as being overwhelmed by the program. An investigation of a complex IBI with interactive worksheets and a high degree of autonomy for people with psychosis is required.

Overall, eHealth approaches have great potential for disseminating evidence-based psychological interventions such as CBTp to people with psychosis. In the field of mobile-based interventions (mHealth), research is yielding convincing evidence of the feasibility, effectiveness, and safety of self-management interventions implemented as smartphone apps (e.g., Ben-Zeev et al., 2014; Kim et al., 2018; Schlosser et al., 2018). However, the use of IBIs as another important modality of eHealth that more closely resembles principles of face-to-face therapy has been less well investigated. Because of the paucity of studies, it is currently unclear whether IBIs are feasible, effective, and safe for people with psychosis. Due to the stronger evidence base related to interventions for positive symptoms (Bighelli et al., 2018) than for negative symptoms (Velthorst et al., 2015), the dissemination of interventions for positive symptoms such as delusions and auditory verbal hallucinations by means of IBIs is particularly promising. The overarching goal of the present study is to evaluate a complex IBI for psychosis in patients with acute positive symptoms such as paranoid ideation and auditory verbal hallucinations. We hypothesized that an IBI based on CBTp—thus, an Internet-based CBTp (iCBTp)—would be used regularly by users with psychosis (feasibility), would reduce the severity of core positive symptoms compared to a waiting period (efficacy), and would be accompanied by only infrequent adverse events and aversive experiences (safety). To test these hypotheses, we conducted an RCT that compared an 8-week iCBTp intervention with a wait-list control group. To our knowledge, this is the first study that has tested a symptom-oriented, CBT-based, complex, guided, Internet-based self-help intervention for psychosis (study protocol: Ruegg, Moritz, Berger, et al., 2018).

Method

Recruitment and Sample Characteristics

We recruited participants via a study website listed in search engines. Additionally, the website was advertised with entries in online forums for people with mental health problems, Internet ads, e-mails to former research participants who wished and consented to be informed about new studies, and flyers sent to inpatient clinics in Switzerland and Germany. The study website informed potential participants about the study aim and procedures, data security provisions, and emergency contacts in case of acute distress as well as contact details for the study team (mail, e-mail, and phone) and provided a link to an online survey platform. Using this platform, we delivered detailed study information and acquired informed consent.

Inclusion criteria were (a) age of 18 or older, (b) sufficient command of the German language, (c) access to the Internet, (d) informed consent, (e) acute and/or lifetime diagnosis of a schizophrenia spectrum disorder according to the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013), (f) simultaneous pharmacological and/or regular psychiatric or psychological care (for reasons of safety), and (g) at least mild severity of at least one of the three symptoms—delusions (P1), hallucinations (P3), or suspiciousness/ persecution (P6)—of the Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, 1987). Criteria for exclusion were (a) acute suicidality, (b) posing an acute danger for others, (c) a neurological disease of the central nervous system that needed to be treated, and (d) unwillingness to formulate an emergency plan that listed contacts for acute crises (e.g., a local psychiatrist). The participants did not receive any financial reimbursement.

Enrollment and Sampling Procedures

Immediately after acquiring informed consent, we assessed the sociodemographic data and the self-report part of the baseline assessment. The survey platform deploying the questionnaires automatically checked inclusion criteria when possible (e.g., age under 18 years). After preliminary evaluation by a study coordinator of the data provided in the survey, an independent assessor contacted interested persons to conduct the clinician-administered part of the baseline assessment via a telephone interview. To establish diagnoses according to the DSM-5 (American Psychiatric Association, 2013), interviewers conducted the Mini International Neuropsychiatric Interview (MINI; Lecrubier et al., 1997)

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


using a version adapted for DSM-5 criteria. This adapted version adhered to DSM-5 criteria but discarded bizarreness of symptoms while considering previous negative symptoms, loosening of associations, and so forth. The independent assessors also collected the necessary data to rate all PANSS items except for G5, “mannerisms and posturing,” which is not possible via telephone. The assessors had a bachelor’s degree in psychology or higher and were trained in workshops that lasted approximately 8 hr and included a theoretical introduction as well as practical exercises with feedback (e.g., role playing an interview via telephone). During the study, if assessors had diagnostic or other questions, they consulted by phone or in person with experienced members of the study team who were also blind to the study condition. To establish interrater reliability, raters assessed an audiotaped interview after the training. The intraclass correlation coefficient (ICC; two-way mixed, single measures, absolute agreement) as an index of interrater reliability was 0.82 (95% confidence interval: 0.73 to 0.89), which is good to excellent according to Cicchetti (1994). As the final step in the enrollment process, inclusion and exclusion criteria were checked by study staff based on self-report and rater-assessed data, and interested persons were either included and randomized or excluded. Excluded participants received a written self-help booklet and were supported in finding a local psychiatrist or psychotherapist if needed, or—if the only criterion for exclusion was subthreshold acute positive symptom severity— were granted access to the self-help independently of this study.

In the recruitment from December 2016 to May 2018, a total of 7,237 persons visited the study website (see Figure 1). Of those, 10% gave informed consent (n = 746). The most frequent reason for not continuing in the enrollment process after providing informed consent, which occurred 606 times (81%), was not completing the questionnaires (n = 507; 84%). Eventually, 101 participants fulfilled all criteria and were randomly assigned to the two study arms. The only reason for exclusion at the last step was positive symptom severity below the cutoff (n = 39). The randomization list was generated with a randomization service (www.random.org) and concealed from the investigators.

Study Design

This RCT compared an active intervention group (iCBTp) with a wait-list control group (WL) in a sample of individuals with psychosis who received care as usual. The intervention group had access to an Internet-based self-help platform and an optional smartphone app. After an assessment at the end of the eighth week after randomization (post assessment), the participants in the waitlist condition also were given access to the self-help platform and the app (see Figure 1). Thus, the follow-up assessments after 6 months were not part of the RCT design. We registered the trial (clinicaltrials.gov, NCT02974400, November 28, 2016) and received ethical approval from the Ethics Committee of the Canton of Bern, Switzerland (KEK 03/14) and the German Psychological Society (SM052015_CH).

Clinical Intervention

Several CBT models and interventions served as the basis for the self-help content of the iCBTp program, which is organized in 11 modules: introduction, paranoid ideation, voice hearing, selfesteem, sleep hygiene, metacognition, depression, mindfulness, worrying, social competence, and relapse prevention (for details, see Supplemental Table A1). Most of the modules address single symptoms such as paranoid ideation or worrying rather than disorders, in accordance with recent trends in research and treatment oriented at individual symptoms (e.g., Garety & Freeman, 2013). Because the introductory module imparts general CBT concepts and techniques such as the ABC protocol (activating event, belief, consequences) that are relevant to all other modules, it was mandatory and was shown to participants at the beginning of the intervention. In addition, participants were instructed to work through the relapse prevention module after completing the other modules of interest. The nine remaining modules (see Supplemental Table A1) addressed psychotic symptoms directly (paranoid ideation and auditory verbal hallucinations) or indirectly by changing processes that facilitate psychotic symptoms according to basic clinical psychology research (e.g., Freeman & Garety, 2014). Because many people with psychosis seem to be motivated to avoid being patronized (Westermann et al., 2015), participants were free to determine the order of the modules (with the exception of the first and last) in order to support positive experiences of autonomy and minimize dropout. Completion of the full set of 11 modules during the intervention period was not expected. Instead, an average frequency of one module per week was defined as full adherence (thus, eight modules during the intervention period).

The content of the modules consisted of texts, illustrations, explanatory videos, and interactive worksheets embedded in the module pages that were called inline worksheets. Typically, a module starts with a textual description of its target (e.g., hearing voices that cause distress) and normalizing information (e.g., occurrence of hearing voices in the general population), as well as a brief “case example” in which a (fictional) affected individual is characterized. Next, a CBT model of maintenance factors is introduced with an explanatory video in most modules (e.g., vicious cycle of thoughts, emotions, and safety as well as other behaviors that facilitate distress due to hearing voices). The model is not stated as fact but as one way of explaining the maintenance of the module target from a psychological perspective, and users are encouraged to critically examine whether they find the model plausible. If it seems plausible or if the user is open to giving it a try, leverage points for psychological interventions are derived from the model (e.g., ABC protocols to address distress-enhancing thoughts related to voice hearing). In the next sections of the module, the individual interventions are explained, and users can often try them out with inline worksheets that, for example, are located between two textual paragraphs on a page. Besides textual worksheets (e.g., ABC protocols), modules may include lists (e.g., a list of positive activities to select from), audio files (e.g., mindfulness meditation), and exercises (e.g., from the metacognitive training; Moritz, Andreou, et al., 2014). Lastly, the main points of the module are summarized, and users can write feedback and ask open questions in a text field, which are later addressed by the guide. A typical module consists of 21 pages (i.e., webpages) and is expected to take about 30 to 60 min to complete. Users are not required to complete a module in a single session, and they can also reaccess pages of previous modules. Each time a participant finishes a module, the appropriate stand-alone worksheet is unlocked in the dashboard of the self-help platform, and users are encouraged to practice with the worksheet (e.g., ABC protocols).

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.



This is comparable to homework assignments in face-to-face therapy. Examples of stand-alone worksheets are the ABC protocol (introduction) and a list of personal strengths and realizations of those strengths in daily life (self-esteem). The modules could be repeated any number of times during the 8-week intervention period as well as afterward. A smartphone app was provided to facilitate the transfer from the program to experience and behavior in daily life. The use of the smartphone app was optional to avoid its posing a barrier (e.g., requiring possession of a smartphone) to the treatment. The app included the worksheets that were also available via the self-help platform. After randomization, participants received an e-mail with login details for the self-help platform. At first login, participants were instructed to choose a username and viewed an introductory video explaining the user interface and the principles of the self-help platform (screencast with audio comment).

The self-help intervention was guided using a secure messaging system within the self-help platform. Guides had at least a bachelor’s degree in psychology and were supervised by an experienced CBT therapist fortnightly. At least once a week, guides checked participants’ progress (modules started, worksheet entries, etc.) and gave written feedback. Additionally, participants were able to write messages to guides or to reply to messages from guides on their own initiative. If participants did not log in for

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


more than a week, guides wrote individualized reminder messages that were delivered via regular e-mail. In their first message to a new user at the beginning of the intervention period, guides introduced themselves, described their role (e.g., providing feedback on worksheets), and emphasized their openness to receiving messages from the user. Only users who requested it were oriented to the program by a guide.

Data security was achieved by several means. First, the communication between clients’ devices (personal computers and/or app on smartphones) and the server was SSL encrypted. Second, access to the platform was only possible with a unique username and password. Third, the self-help system was free of any personal data. Thus, personal data such as names or e-mail addresses were neither requested nor allowed in the platform. Lastly, e-mail addresses, telephone numbers, and names of study participants were not stored digitally but on a handwritten paper list locked in a cabinet to further increase data security. Reminder e-mails, for example, were sent by manually replacing the participant code with the e-mail address from the paper list, and the e-mail text was not stored together with the e-mail address.

Outcome Measures

The German translations of the self-report scales and rater-administered assessments that are used as primary and secondary outcomes in this study are established research instruments.

Primary outcomes. The positive factor of the PANSS, which measured the overall severity of positive symptoms (PANSS-PF; van der Gaag, Hoffman, Remijsen, Hijman, & Wiersma, 2006), the Launay-Slade Hallucination Scale, which measured hallucinatory experiences (LSHS; Launay & Slade, 1981; German version: Lincoln, Keller, & Rief, 2009), the Paranoia Checklist, which measured paranoid ideation (PC; Freeman et al., 2005), and the MINI item that indicates the presence of acute psychotic symptoms served as the primary outcomes of this RCT.

The positive factor of the PANSS, based on van der Gaag et al. (2006), is scored with the formula delusions (P1) + hallucinations (P3) + suspiciousness/persecution (P6) + grandiosity (P5) + somatic concern (G1) + unusual thought content (G9) + lack of judgment and insight (G12) + active social avoidance (G16) -difficulty in abstract thinking (N5) and has a potential range from 1 to 55. The ICC for the positive factor was 0.84 (95% confidence interval: 0.70 to 0.95; two-way mixed, single measures, conservative absolute agreement), which is excellent according to Cicchetti (1994). The raters were kept blind regarding the study group allocation. To check blindness, raters guessed the study group after each interview (with the exception of the initial interviews, which took place before the actual randomization) and were not able to guess the group, x2(3) = 1.33, p = .72. The self-report scale LSHS has adequate psychometric properties with an internal consistency of Cronbach’s a = .79 (Lincoln et al., 2009; present study: a = .87), a Likert scale response ranging from certainly does not apply to me = 0 to certainly applies to me = 4, and a score ranging from 0 to 48. The PC consists of three subscales that measure the frequency of paranoid thoughts, the conviction, and the associated distress. The PC subscales have an excellent reliability of Cron-bach’s alpha of 0.90 or above (Freeman et al., 2005; present study: as > 0.95) and good validity (e.g., correlation of subscales with PANSS persecutory delusions [P6] approximately r = .50, Lincoln, Ziegler, Lullmann, Muller, & Rief, 2010; correlation with P6 in present study: frequency r = .51, p < .001, conviction: r = .53, p < .001, distress: r = .36, p < .001). By adding the PC subscale scores, a total score was built that was not normally distributed and thus was transformed with the natural logarithm.

We computed a composite score of the three continuous primary outcomes to avoid alpha inflation due to multiple testing. First, the mean score at baseline of each outcome was subtracted from the respective outcome score at all measurement times. Second, the values at all measurement times were divided by the standard deviation of the respective outcome at baseline. Finally, the three standardized values were averaged to build the composite score. The composite was only computed if at least two of the three outcome measure were present. The test-retest reliability of the composite score based on the correlation between baseline and postwaiting scores in the WL condition was r = .78 (p < .001) and thus was adequate.

Secondary outcomes. For each module except for the introduction and the module on relapse prevention, we assessed a questionnaire that measured a related construct (e.g., depression severity for the module on depression; see Supplemental Table A1 for a complete list of the questionnaires). Four general outcomes were also part of the assessments: (a) motivational incongruence (brief version of the incongruence questionnaire; Grosse Holtforth, 2008), which reflects the dissatisfaction or violation of motives such as autonomy, (b) psychological quality of life (WHOQOL Group, 1998), (c) internalized stigma (Boyd Ritsher, Otilingam, & Grajales, 2003), and (d) satisfaction with the iCBTp intervention (Schmidt & Wittmann, 2002). Due to limited space, the psychometric properties of the secondary outcomes based on independent samples are only reported in the study protocol (Ruegg, Moritz, Berger, et al., 2018). However, their reliabilities in the current sample are reported in Table 1. Negative experiences with and negative effects of the program were assessed using the Questionnaire about Side Effects Psychosis and Internet (QueSPI; Ruegg, Moritz, Berger, et al., 2018). The psychometric evaluation of the German version of the Delusion and Voices Self-Assessment Scale (DV-SA; Pinto, Gigantesco, Morosini, & La Pia, 2007), which was also part of the assessment, will be reported in a separate publication. Similarly, the Box Task (Moritz, Goritz, et al., 2017) as well as potential predictors, process measures, and moderators such as the Attitudes toward Psychological Online Interventions questionnaire (APOI; Schroder, Sautier, Kriston, Berger, & Moritz, 2015) were part of the assessment but will be reported elsewhere.

Power Calculation

We conducted the power calculation for this trial using G*Power 3 (Faul, Erdfelder, Lang, & Buchner, 2007). Medium-sized or larger effects f > 0.25) were deemed clinically significant and used for the calculations. A sample size of 128 was computed to detect a medium-sized effect using an ANCOVA with one covariate assuming a power of 80% and an a-level of 5%. To compensate for an expected dropout rate of 10%, we increased the target sample size to 140. A post hoc sensitivity analysis with the actual total sample of 101 showed that effects of f = 0.25 or larger—corresponding to Cohen’s d = 0.56—could be detected in

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


Table 1

Sample Characteristics at Baseline as Well as Program Usage and Condition

Characteristic

iCBTp (n = 50)

WL (n = 51)

Statistics

Test

P

Sociodemographic

Sex (male/female)

16/34

26/25

X2 = 3.744

.053

Age

39.3 (10.0)

40.8 (9.2)

t(99) = - 0.745

.458

Family status (long-term relationship/single/divorced/unknown)

13/17/1/19

13/16/1/21

X2 = .120

.989

Years of education

12.0(1.3)

11.4 (1.8)

t(99) = 1.657

.101

Education: Highest level (low/middle/high/unknown)

2/9/20/19

1/8/21/21

X2 = .507

.917

Monthly income (<750€/<1500€/<2500€/>2500€/unknown)

Clinical

6/16/6/3/19

7/16/6/1/21

X2 = 1.167

.883

Main diagnosis (schiz/schizaff/delus/unspec)

44/3/3/0

48/1/1/1

X2 = 3.164

.367

Years of psychosisa

11.3 (7.4)

15.0 (8.3)

t(98) = - 2.320

.022

Number of hospitalizations

7.5 (10.5)

7.0 (6.1)

t(88) = 0.180

.792

PANSS positive scale

15.7 (4.7)

14.8 (4.7)

t(99) = 0.915

.362

PANSS negative scale

12.9 (5.2)

12.5 (4.2)

t(99) = 0.458

.648

PANSS general psychopathologyb

31.4(9.4)

30.4 (7.4)

t(99) = 0.581

.564

PANSS positive factorc

16.9 (6.0)

15.7 (5.2)

t(99) = 1.020

.310

Paranoia Checklist-Frequencyd

2.5 (1.0)

2.2 (0.9)

t(99) = 1.444

.152

Paranoia Checklist-Convictiond

2.5 (1.1)

2.1 (1.0)

t(99) = 1.778

.078

Paranoia Checklist-Distressd

2.5 (1.1)

2.2 (1.1)

t(99) = 1.249

.215

Launay Slade Hallucination Scale Treatment

17.3 (10.5)

15.1 (11.4)

t(99) = 0.999

.320

Antipsychotic medication (yes/no)

41/9

44/7

X2 = .346

.556

Percentage of maximum antipsychotic dose

37.4 (37.71)

46.3 (42.8)

Z = - 0.996

.319

Psychological interventions lifetime (psychotherapy/MCTe)

36(30/11)

37 (34/17)

X2 = .004

.951

Self-help lifetime (group/book/online/forum) Program usagef,g

27 (11/9/5/12)

27 (17/7/5/11)

X2 = .011

.915

Completed modules

4.0 (8.3)

2.0 (5.0)

Z = - 2.372

.018

At least one login (yes/no)

49/1

41/10

X2 = 8.066

.005

Number of logins

14.0 (33.5)

11.0 (19.0)

Z = -1.571

.116

Time spent in program (minutes)

262.2 (608.3)

131.6(424.7)

Z = - 2.101

.036

Worksheets used

6.0 (9.3)

2.0 (6.0)

Z = - 2.324

.020

Messages sent to moderator

2.0 (4.0)

1.0 (3.0)

Z = -1.809

.070

Messages received from moderator

7.0 (7.5)

6.0 (6.5)

Z = -1.162

.245

Smartphone app used (yes/no)

16/34

7/44

X2 = 4.794

.029


Note. iCBTp = Internet-based CBT for psychosis; WL = wait-list control; PANSS = Positive and Negative Syndrome Scale.

a Incomplete data, due to technical problems, in the variable “years of psychosis” was imputed by years since first hospitalization. b Because the PANSS item G5 “mannerisms and posturing” could not be assessed via telephone, the general psychopathology scale of the PANSS was extrapolated with the formula ((sum of scale)/15)*16. c Score computed in line with van der Gaag et al. (2006). d Test statistics are based on logarithm-transformed values and descriptives on raw values. e Metacognitive training (Moritz, Andreou, et al., 2014). f Data for the WL group refer to the program usage after the waiting period. g Median and interquartile ranges are reported instead of mean and standard deviations.


this study. According to this calculation, the power of this study was sufficient to detect medium to large-sized effects.

Statistical Analyses

We conducted mixed-model analyses with the between-subjects factor condition (iCBTp vs. WL group) and the within-subject factor time (baseline vs. post). Mixed-model analyses, which include all observed and thus also partial data without imputation and are in line with the intention-to-treat (ITT) principle, are advantageous as they result in valid and unbiased estimates in case of data missing at random (Bell, Fiero, Horton, & Hsu, 2014). To test the hypotheses, we used the interaction term Time X Condition. We analyzed the composite score and subsequently the single primary outcomes as well as the secondary outcomes using mixed-effect models with unstructured covariance matrices and restricted maximum likelihood estimation. In these models, timepoints were nested within participants. The mixed-effect models and all other analyses were computed with SPSS Version 26.0.0.0 for Windows


(SPSS Inc., Chicago, Illinois). All tests were two-sided. In addition to the ITT sample, we also analyzed the participants who used the intervention per protocol (PP) by completing at least four of the optimal number of eight modules. Effect sizes were computed based on the estimated means and standard errors of the mixed-effect model analyses. The between-subjects effect sizes and their confidence intervals were baseline corrected. Thus, the mean of the baseline score was subtracted from the mean of the post score, and the standard deviation of the difference was calculated taking into account the correlation of baseline- and postscores,1 based to Morris (2008). Effect sizes are reported with a 95% confidence interval. In line with Jacobson, Follette, and Revenstorf (1984), we defined clinically significant improvement (CSI) in the three primary outcomes PANSS-PF, PC, and LSHS as (a) improvement based on the reliable change index (RCI; Jacobson et al., 1984) and


Mcorrected         Mpost Mbaseline, and SDcoirected          sqrt(SEpre

SE2>st_SEpre* SEpost* Tpre,post) * sQrt(N).


This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


(b) belonging to a functional population after the study period of 8 weeks. Sources for the normative data are reported in Supplemental Table B1. Additionally, odds ratios for CSI in the intervention group compared to the wait-list group are reported with 95% confidence intervals. Odds ratios greater than one indicate that the iCBTp intervention is accompanied by higher odds of clinically significant improvement than the WL.

Results

Baseline Characteristics and Dropout Analysis

Participants in the iCBTp and WL conditions did not significantly differ in sociodemographic, clinical, or treatment-related characteristics (see Table 1), with the exception of a longer duration of psychosis in the WL condition. Thus, the randomization did not balance out systematic differences between the two groups in the duration of the illness. Also, differences in the gender ratio between the two groups bordered statistical significance (iCBTp: 68% female, WL: 49% female; p = .053). To account for these differences, we repeated the analyses of outcomes with the covariates “gender” and “years of psychosis,” resulting in essentially identical results. Most participants were diagnosed with schizophrenia (91%), and the positive symptom severity was comparable to face-to-face outpatient CBTp trials (e.g., M = 15.25 in this study vs. M = 14.95 in Lincoln et al., 2012). A slight majority of the participants in the iCBTp condition completed four or more modules and met the criteria for per protocol analyses (52%); 32% did not complete a single module.

During the 8-week period after randomization, 13 participants (29%) in the WL condition and 19 participants (45%) in the iCBTp condition were in simultaneous psychological therapy, x2(1) = 2.497, p = .114. In the WL and iCBTp conditions, four (9%) and five (12%) participants, respectively, were in inpatient treatment during the 8 weeks, x2(1) = 0.213, p = .644. The distribution of medication changes in the WL (increase: 7, decrease 9; n = 46) and the iCBTp condition (increase: 3, decrease, 10; n = 45) did not differ, x2(2) = 1.706, p = .426.

Twelve of the randomized participants (12%) did not complete the assessment either post intervention or post waiting period (iCBTp: n = 7, WL: n = 5) despite reminders and thus were considered dropouts. The frequency of dropping out did not differ between conditions, x2(1, N = 101) = 0.425, p = .515, and dropouts did not differ in sociodemographic or treatment-related variables from completers, ps > .05. On the composite score and the single primary outcomes PC, LSHS, and MINI acute psychosis, there were no significant differences between completers and dropouts, p > .10. However, on the PANSS-PF, the dropout group had significantly higher scores (M = 19.42, SD = 5.96) than the completer group (M = 15.89, SD = 5.44), t(99) = — 2.085, p = .040, d = 0.642. Thus, participants who dropped out had higher positive symptom severity at baseline.

Primary Outcomes

In line with our hypothesis, the composite score of the primary outcomes decreased more strongly from baseline to post assessment in the iCBTp condition than in the WL condition, according to the significant interaction Time X Condition in the ITT analysis (see Table 2), F(1, 87.28) = 4.04, p = .047. The effect size of the difference between groups at post assessment was small to medium-sized, dbetween = — 0.37, 95% CI [—0.67, —0.07]. Due to the significant interaction of Time X Condition with the composite score as the dependent variable, subsequent separate analyses of the individual primary outcomes as dependent variables were justified. The interaction was also significant for the individual primary outcome LSHS, F(1, 88.22) = 7.15, p = .009, and met the Bonferroni Holm correction for alpha inflation (pcorr = 0.017 > 0.009), showing that the iCBTp condition was accompanied by a greater reduction in hallucination severity compared to the WL condition, dbetween = 0.33, 95% CI [—0.06, 0.72]. Contrary to our hypothesis, the interaction Time X Condition was neither significant for the individual outcome PC nor for PANSS-PF, Fs < 3.07, ps > .083. Regarding acute psychosis according to the MINI, 20% improved from baseline to post assessment, 76% remained stable, and 4% deteriorated in the iCBTp condition. In the WL condition, 18% improved, 80% did not change, and 2% deteriorated. The group differences are not significant, x2(2) = 0.490, p = .783.

The analyses were repeated based on the PP sample that included 26 participants of the iCBTp group who had completed at least four modules as well as the entire WL group. The interaction term Time X Condition was significant with a medium-sized between-groups effect size for the composite score as dependent variable, F(1, 68.02) = 8.68, p < .01, dbetween = 0.43. The interaction was also significant for the single primary outcomes LSHS, F(1, 68.10) = 16.34, p < .001, dbetween = 0.54, and PC, F(1, 69.04) = 5.63, p < .05, dbetween = 0.41, and met the Bonferroni Holm correction for alpha inflation (pcorr = 0.017). The interaction term was not significant for the PANSS positive factor, F(1, 69.02) = 0.02, p > .05, dbetween = 0.02. Regarding acute psychosis according to the MINI in the PP sample, 23% improved from baseline assessment to post assessment, 73% did not change, and 4% deteriorated in the iCBTp group. The differences between the groups were not significant, x2(2) = 0.615, p = .735.

Secondary Outcomes

The ITT analyses of the secondary outcomes resulted in a significant interaction of Time X Condition for mindfulness (MAAS), F(1, 88.46) = 8.00, p < .01, dbetween = -0.34, but no significant effect for any other of the 11 outcomes, Fs < 2.48, ps > .12, ldbetweenl < 0.21 (see Table 2). The PP analyses revealed significant interactions of Group X Time for the four secondary outcomes self-esteem (RSE), mindfulness (MAAS), social skills (ICQ), and psychological quality of life (WHOQOL-BREF psychological), Fs > 4.33, ps < .05, with small to medium-sized between-groups effect sizes in favor of iCBTp, ldbetweenl > 0.32. However, none of those effects were significant after correction for alpha error inflation with the Bonferroni Holm method (ps > pcorr = 0.05/11 = 0.005).

Clinically Significant Improvements

CSI as defined by Jacobson et al. (1984) is reliable change and belonging to a functional population at post assessment. In the iCBTp group, 33% of the participants showed CSI in LSHS in contrast to 13% in the WL group (see Supplemental Table B1),

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


Table 2

Estimated Baseline and Post-Wait/-Treatment Scores of the Outcomes as Well as Effect Sizes in Intention-to-Treat and Per-Protocol Samples

Intention-to-Treat sample                                                                                      Per-Protocol sample

Outcome8

Baseline3

Postb

Effect size (ES) d [95% CI]

Follow-up

Baseline®

Postf

ES d [95% CI]

M(SD)

M (SD)

Within (Pre/Post)

Between

Statistic0

es

M (SD)

M (SD)

Within (Pre/Post)              Between

Primary outcomes

Composite score

Interaction effect:

. F(l, 87.28) = 4.04, p =

.047*

Interaction effect: F(l, 68.02) = 8.68, p = .004*

iCBTp

0.12(0.76)

-0.26 (0.84)

-0.82 [-1.23, -0.41]

0.24 [-0.15, 0.63]

F = 9.59

-0.37

-0.03 (0.76)

-0.56(0.73)

-0.99 [-1.57, -0.42]       0.43 [-0.05, 0.90]

WL

-0.12(0.75)

-0.28 (0.84)

-0.32 [-0.71, 0.07]

p = .003

[-0.67, -0.07]

-0.12(0.76)

-0.29(0.74)

-0.33 [-0.72, 0.06]

LSHS (hallucin., a = .87)

Interaction effect:

F(l, 88.22) = 7.15, p =

.009*

Interaction effect: F(l, 68.10) = 16.34, p < .001*

iCBTp

17.26 (10.97)

12.60 (10.95)

-0.56 [-0.96, -0.16]

0.33 [-0.06, 0.72]

F = 6.14

-0.30

17.73 (11.38)

10.58 (10.50)

-0.88 [-1.45, -0.31]       0.54 [0.06, 1.02]

WL

15.08 (10.97)

14.42(10.83)

-0.11 [-0.49, 0.28]

p = .016

[-0.59, 0.00]

15.08(11.38)

14.42(10.69)

-0.10 [-0.49, 0.29]

PC (paranoia, as > .95)

Interaction effect:

F(l, 87.85) = 3.07, p =

.083

Interaction effect: F(l, 69.04) = 5.63, p = .020*

iCBTp

0.82 (0.42)

0.64 (0.46)

-0.63 [-1.04, -0.23]

0.24 [-0.15, 0.63]

F = 8.30

-0.22

0.74 (0.42)

0.49 (0.40)

-0.86 [-1.42, -0.29]       0.41 [-0.07, 0.88]

WL

0.69 (0.42)

0.63 (0.46)

-0.13 [-0.52, 0.26]

p = .005

[-0.51,0.08]

0.69 (0.42)

0.63 (0.41)

-0.13 [-0.52, 0.25]

PANSS-PF (ICC = .84)

Interaction effect:

F(l, 88.38) = 0.16, p =

.686

Interaction effect: F(l, 69.02) = 0.02, p = .893

iCBTp

16.88 (5.59)

14.84 (6.17)

-0.77 [-1.18, -0.37]

0.05 [-0.34, 0.44]

F = 3.35

-0.32

15.08 (5.11)

13.19(5.46)

-0.68 [-1.24, -0.12]       0.02 [-0.45, 0.49]

WL

15.75 (5.59)

14.00 (6.19)

-0.43 [-0.82, -0.03]

p = .071

[-0.61, -0.02]

15.75 (5.11)

13.98 (5.60)

-0.47 [-0.86, -0.08]


Secondary outcomes

PANSS negative (a

= .82)

Interaction effect: F(l, 92.25) = 0.21, p =

.646

Interaction effect: F(l, 70.62) = 0.37, p = .544

iCBTp

12.92(4.72)

12.03 (4.54)       -0.39 [-0.79, 0.00]

0.06 [-0.33,0.45]

F =

.38

0.07

12.31 (4.95)

11.23 (4.51)      -0.58 [-1.13,-0.02]

0.09 [-0.38,0.56]

WL

12.49 (4.71)

11.90 (4.56)       -0.14 [-0.53, 0.25]

F =

.542

[-0.23,0.36]

12.49 (4.95)

11.89(4.63)      -0.13 [-0.52, 0.26]

PANSS general (a =

= .79)

Interaction effect: F(l, 87.83) = 0.00, p =

.994

Interaction effect: F(l, 68.70) = 0.03, p = .873

iCBTp

30.40 (7.93)

Z13\ (7.79)      -0.74 [-1.14, -0.33]

0.00 [-0.39, 0.39]

F =

6.00

-0.26

26.96 (7.12)

23.69(6.71)      -1.07 [-1.65,-0.49]

0.02 [-0.45, 0.49]

WL

29.48 (7.93)

26.40 (7.82)       -0.52 [-0.91, -0.12]

F =

.017

[-0.56, 0.03]

29.48 (7.12)

26.38 (6.86)      -0.58 [-0.97, -0.18]

RSE (self-esteem, a

= .92)

Interaction effect: F(l, 90.14) = 1.72, p =

.193

Interaction effect: F(l, 70.19) = 4.33, p = .041

iCBTp

24.86 (8.05)

26.87 (7.71)        0.38 [-0.02, 0.77]

-0.16 [-0.55, 0.23]

F =

4.78

0.24

26.42 (7.73)

29.65(6.87)        0.54 [-0.02, 1.09]

-0.32 [-0.79, 0.16]

WL

26.77 (8.06)

27.38 (7.63)        0.12 [-0.27, 0.51]

F =

.032

[-0.06, 0.53]

26.77 (7.73)

27.41 (7.03)        0.13 [-0.26,0.52]

ISI (insomnia, a =

.87)

Interaction effect: F(l, 89.66) = 0.05, p =

.822

Interaction effect: F(l, 70.78) = 0.98, p = .325

iCBTp

12.64(6.75)

11.01 (7.07)       -0.30 [-0.70, 0.09]

0.03 [-0.36, 0.42]

F =

.02

-0.02

12.65 (6.74)

10.00 (6.02)      -0.49 [-1.05, 0.06]

0.17 [-0.30,0.65]

WL

10.77 (6.76)

9.40 (6.97)       -0.23 [-0.62, 0.16]

F =

.886

[-0.31,0.28]

10.77 (6.73)

9.41(6.21)       -0.23 [-0.62, 0.16]

PSWQ (worrying, a

= .96)

Interaction effect: F(l, 88.90) = 0.05, p =

.823

Interaction effect: F(l, 70.23) = 0.35, p = .557

iCBTp

28.00 (14.04)

25.21 (14.56)     -0.27 [-0.66, 0.13]

0.03 [-0.36, 0.42]

F =

12.37

-0.39

25.85(13.45)

21.92(13.56)     -0.31 [-0.86,0.23]

0.10 [-0.37,0.57]

WL

23.96 (14.03)

21.64(14.40)     -0.24 [-0.63, 0.15]

F =

.001

[-0.69, -0.10]

23.96(13.45)

21.59 (13.95)      -0.26 [-0.65,0.13]

PHQ (depression, a

= .83)

Interaction effect: F(l, 86.43) = 0.07, p =

.795

Interaction effect: F(l, 67.36) = 0.14, p = .711

iCBTp

13.00 (6.27)

11.64(6.31)       -0.27 [-0.66, 0.13]

-0.04 [-0.43,0.35]

F =

3.37

-0.21

11.50 (6.11)

9.35 (5.58)       -0.39 [-0.94, 0.16]

0.07 [-0.41,0.54]

WL

11.53 (6.28)

9.89 (6.23)       -0.29 [-0.68, 0.10]

F =

.071

[-0.50, 0.08]

11.53 (6.11)

9.85 (5.76)       -0.30 [-0.69, 0.09]

MAAS (mindful., a

= .90)

Interaction effect: F(l, 88.46) = 8.00, p =

.006

Interaction effect: F(l, 69.89) = 7.65, p = .007

iCBTp

3.45 (0.93)

3.82(0.98)         0.64 [0.23, 1.04]

-0.34 [-0.73,0.05]

F =

1.62

0.16

3.33 (0.95)

3.76 (0.97)         0.68 [0.12, 1.24]

-0.39 [-0.87, 0.08]

WL

3.79 (0.93)

3.80 (0.96)        0.02 [-0.37, 0.41]

F =

.208

[-0.13,0.45]

3.79(0.95)

3.80 (0.99)        0.02 [-0.37, 0.41]

ICQ (social skills, a

= .81)

Interaction effect: F(l, 85.71) = 1.53, p =

.220

Interaction effect: F(l, 68.04) = 5.38, p = .023

iCBTp

4.58 (1.70)

4.97 (1.87)        0.30 [-0.09, 0.70]

-0.16 [-0.55, 0.23]

F =

.48

0.08

4.47(1.61)

5.23 (1.68)         0.55 [0.00, 1.11]

-0.37 [-0.84, 0.11]

WL

5.38 (1.70)

5.44(1.86)        0.05 [-0.33, 0.44]

F =

.490

[-0.22, 0.37]

5.38(1.61)

5.44(1.73)        0.05 [-0.34, 0.44]

K-INK (incongr., a

= .90)

Interaction effect: F(l, 84.92) = 1.48, p =

.228

Interaction effect: F(l, 66.98) = 2.17, p = .146

iCBTp

3.16(0.74)

2.86 (0.84)       -0.52 [-0.92, -0.12]

0.17 [-0.22, 0.56]

F =

5.24

-0.27

2.86 (0.72)

2.50 (0.78)       -0.67 [-1.23,-0.11]

0.25 [-0.23,0.72]

WL

2.98 (0.74)

2.84(0.84)       -0.21 [-0.60, 0.18]

F =

.025

[-0.56, 0.03]

2.98 (0.72)

2.84(0.80)       -0.22 [-0.61, 0.17]

Psychol. QoL (a =

.86)

Interaction effect: F(l, 85.52) = 2.48, p =

.119

Interaction effect: F(l, 66.67) = 6.39, p = .014

iCBTp

10.80 (3.39)

11.74(3.63)        0.39 [-0.01,0.78]

-0.21 [-0.60, 0.19]

F =

7.62

0.30

11.44 (3.24)

13.03 (3.38)        0.67 [0.11, 1.23]

-0.39 [-0.86, 0.09]

WL

12.20 (3.39)

12.32(3.61)        0.05 [-0.34, 0.43]

F =

.007

[0.00, 0.59]

12.20 (3.24)

12.32(3.47)        0.05 [-0.34, 0.44]

ISMI (stigma, a = .

.81)

Interaction effect: F(l, 84.87) = 2.22, p =

.140

Interaction effect: F(l, 67.11) = 3.05, p = .085

iCBTp

2.52 (0.59)

2.37 (0.62)       -0.37 [-0.76, 0.03]

0.19 [-0.21,0.58]

F =

10.27

-0.36

2.41 (0.60)

2.22 (0.59)       -0.56 [-1.12, -0.01]

0.24 [-0.23, 0.71]

WL

2.27 (0.59)

2.24 (0.62)       -0.06 [-0.45, 0.32]

F =

.002

[-0.65, -0.06]

2.27(0.61)

2.24(0.61)       -0.06 [-0.45, 0.33]

Note. iCBTp :

= Internet-based CBT for psychosis; WL = wait-list control; LSHS = Launay-Slade Hallucination Scale; PC

= Paranoia Checklist; PANSS-PF = positive factor of the PANSS;


PANSS = Positive and Negative Syndrome Scale; RSE = Rosenberg Self-Esteem Scale; ISI = Insomnia Severity Index; PSWQ-A = Penn State Worry Questionnaire abbreviated; PHQ = Patient Health Questionnaire; MAAS = Mindful Attention and Awareness Scale; ICQ = Brief Interpersonal Competence Questionnaire; K-INK = Brief version of the Incongruence Questionnaire; incongr. = incongruence; Psychol. QoL = Psychological Quality of Life; ISMI = internalized stigma of mental illness.

a Sample size iCBTp group n = 50, WL group n = 51. b For the exact sample sizes, see Supplemental Table DI. CF- and p values of the main effect of time (post-intervention vs. 6-month follow-up). d Cohen’s d of the difference between post-intervention and follow-up scores for the entire sample. e Sample size iCBTp n = 26, WL n = 51. f For the exact sample sizes, see Supplemental Table DI. s Reliability of self-reported (Cronbach’s a at baseline) and rater administered outcomes (intraclass correlation coefficients, ICC) in parentheses.

* Significant after alpha correction with the Bonferroni Holm method.


This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


with an odds ratio of CSI for iCBTp compared to WL of 5.07, Z = 2.63, p < .01. The differences in CSI between groups were smaller for the PC (26% CSI in the iCBTp group vs. 11% in the WL group), and the odds ratio only bordered statistical significance, OR = 2.82, Z = 1.76, p = .08. Regarding the PANSS positive factor, the percentage of CSI was low in the iCBTp and the WL group (18% and 24%, respectively; OR = 0.67, Z = 0.77, p = .44).

Treatment Effects at 6-Month Follow-Up

To investigate long-term effects, the assessments directly after the intervention and after the 6-month follow-up period were analyzed with data from both groups of the ITT sample using mixed models with primary and secondary outcomes as dependent variables (see Supplemental Table D1 for means and standard deviations). Whether the access to the intervention was immediate or delayed did not have a significant effect on any of the outcomes (main effect group; all Fs < 2.90, all ps > .09), and the interaction of Time X Condition was not significant for any of the outcomes (all Fs < 2.31, allps > .13). However, the main effect of time was significant for the composite score, the LSHS, the PC, and six of the secondary outcomes, with all effects in the direction of improvement (for details and statistics, see Table 2 and Supplemental Table D1). The effects of the change from post intervention to follow-up ranged from small to medium-sized (0.220 < d < 0.393). There was no significant improvement over time in positive symptom severity measured with the PANSS-PF, F(1, 69.77) = 3.35,p = .071, d = -0.32, 95% CI [-0.61, -0.02]. Thus, the positive effects over the follow-up period of 6 months on paranoia, hallucinatory experiences, positive symptom severity, and all secondary outcomes such as self-esteem increased further or remained stable. None of the outcomes significantly deteriorated during the follow-up period.

Program Usage and Satisfaction

Participants used the intervention platform for a median duration of 4 hr and 4 min (i.e., 04:04; IQR = 07:19; range: 00:00 to 76:17), logged in a median of 12.0 times (IQR = 28.5; range: 0 to 447), sent a median of 1.0 message to a moderator (IQR = 3.5; range: 0 to 25), and completed a median of 3.0 modules (IQR = 6.0; range: 0 to 11) and 5.0 worksheets (IQR = 8.0; range 0 to 21). Thirty-three participants did not complete any modules (33%). The number of reminder e-mails did not differ between the groups, t(99) = 0.310, p = .757 (iCBTp: M = 2.00, SD = 1.641; WL: M = 1.90, SD = 1.540). The optional app was used by 23% of the participants. According to nonparametric tests, the WL group with delayed access to the intervention completed fewer modules, spent less time in the program, used fewer worksheets, sent fewer messages, and installed the app less often (see Table 1) compared to the iCBTp group with immediate access, ps < .05, but there was no significant difference regarding number of logins, Mann-Whitney U = 1044.00, p = .12. For 89% (SD = 26%) of the participants, the average overall satisfaction with the intervention was high. Users were satisfied with the program’s quality (89%), type of help (66%), need orientation (66%), extent of help (68%), and practicality (75%). The majority would recommend the program to a friend (91%), would use the program again (76%), and were generally satisfied with it (88%).

Negative Experiences and Effects

Eleven participants experienced an adverse event during the intervention (11%; iCBTp: n = 3, WL: n = 8). Eight of the events were not related to the study (73%), and none was definitely related to the study (0%). The remaining three adverse events that potentially related to the study (27%) involved a change in antipsychotic medication (n = 2, both in WL) and the experience of a vision while filling in a questionnaire (n = 1, in iCBTp). The number of negative experiences and effects reported by the 80 participants who filled in the corresponding questionnaire (QueSPI) ranged from 0 to 15. Twenty-one percent of the participants reported no negative experiences and effects at all. The most frequent negative experiences and effects were “Human contact was missing in the self-help program via the Internet” (38%; for details see Supplemental Table C1).

Discussion

The present study compared an 8-week, CBT-based, symptom-oriented, guided IBI for people with psychosis to a waiting period in a sample of participants diagnosed with schizophrenia spectrum disorders and acute or only partly remitted positive symptoms who simultaneously received care as usual. As hypothesized, the composite of the three primary outcomes decreased more in the intervention group than in the control group. However, of the three primary outcomes, only self-reported hallucinations decreased significantly more during iCBTp, with a small to medium-sized effect. Beyond that, the PP analysis with participants who completed at least 50% of the planned modules indicated significant effects on the composite score and hallucination severity and additionally on paranoid ideation. Adherence was mixed, with a group of frequent users and another group of nonusers (similar to adherence in depression: Karyotaki, Kleiboer, Smit, Turner, & Cuijpers, 2015). Adverse events, as well as negative experiences and effects, were infrequent, particularly regarding psychotic symptoms. The satisfaction with the program was high.

With a small to medium-sized effect of iCBTp on overall positive symptom severity compared to waiting with care as usual (d = 0.37), the brief intervention can be deemed effective for individuals with acute or partly remitted disorders in the schizophrenia spectrum. Remarkably, this effect size is very similar to the results of a recent network meta-analysis of trials with face-to-face therapies in psychosis (CBT vs. WL: d = 0.36; Bighelli et al., 2018). For most participants, care as usual included antipsychotic medication, demonstrating that a purely Internet-based intervention for psychosis has an add-on effect to the treatment effect of antipsychotic medication. The effect was pronounced for hallucinations and attenuated for paranoid ideation, which is in line with CBT-based pilot trials using web technology that have demonstrated the efficacy of an intervention on auditory verbal hallucinations (residing on a computer in an inpatient setting; Gottlieb et al., 2013) but not of an intervention on paranoia (delivered via the Internet; Ruegg et al., 2018). In contrast to the significant effects measured by self-report scales, the clinician-administered primary outcome of positive syndrome severity did not indicate an effect in the ITT or the PP analyses. This is surprising given the generally high concordance of self-report and clinician-administered scales (Lincoln et al., 2010) and the substantial association of hallucination severity measured with the PANSS P3

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


item and the LSHS as well as paranoid ideation measured with the PANSS P6 item and the PC in this study (r = .51, p < .001, and r = .47, p < .001, respectively). One possible explanation of this inconsistency is a higher sensitivity to change in symptom severity or associated distress of self-report compared to clinician-administered instruments, and another possible explanation is the potential unsuitability of telephone interviews for valid assessment of psychotic symptoms. Taking into account the significant and medium-sized main effect of Time (baseline vs. post), F(1, 88.38) = 27.08, p < .001, d = -0.57, 95% CI [ — 0.85, —0.29], it seems most likely to us that the PANSS via telephone is reliable and valid but not as sensitive to change as the self-report scales, which have been developed to detect differences even in nonclinical populations.

Particularly in the PP analyses of the data from participants who used the program regularly, the intervention had a significant effect on a subset of secondary outcomes that measure participants’ psychological resources (mindfulness, self-esteem, social skills), although those effects did not meet conservative alpha correction. Post hoc it is possible to speculate that the intervention establishes or activates resources. Furthermore, the follow-up effects from post intervention to six months later either indicated further improvement or stability of primary as well as secondary outcomes, and, even descriptively, none of the measures deteriorated according to group-level means. Although these effects have only limited validity due to the transition to a nonrandomized research design in the follow-up period that introduced alternative explanations for change such as spontaneous remission or effects of other treatments, sleeper effects that unfold over time after an intervention are a plausible explanation (Moritz, Veckenstedt, et al., 2014).

The satisfaction with the intervention was high, and a slight majority of the participants used the program per protocol (52%). However, it should be noted that 32% of participants did not complete a single module and 14% were dropouts in the iCBTp condition, which is in line with CBTp trials involving face-to-face contact (e.g., 11% in Lincoln et al., 2012). Furthermore, the participants who dropped out during the trial had higher positive symptom severity at baseline (d = 0.64). These findings provide a starting point for balancing the benefits and disadvantages of complex, demanding interventions with those of simple interventions with low cognitive demands (Rotondi et al., 2015). Individuals with a schizophrenia spectrum disorder who are able (and willing) to use a complex self-help intervention do benefit, according to this study, whereas the intervention seems to be unfeasible for others. Instead of reducing program complexity for all participants to meet the needs of participants with the lowest cognitive capacity, the highest need for personal support, and so on, we advocate tailoring interventions to the requirements and expectations of single individuals or subgroups. For example, in an IBI for depression, individualized feedback yielded a lower attrition rate (Zagorscak, Heinrich, Sommer, Wagner, & Knaevelsrud, 2018), making a similar approach in IBIs for psychosis promising.

Participants in the WL condition with delayed access to the IBI were less active users (e.g., completed fewer modules; see Table 1) than the participants in the iCBTp condition. The possible explanation that the guides were less engaged in the WL condition is unlikely because they wrote as many messages to WL as to iCBTp participants (see Table 1). However, reduced motivation of participants with delayed access due to reduced symptom severity after waiting for 8 weeks is a plausible explanation for this finding. If this is the case, IBIs should be offered without delays to individuals with psychosis interested in this approach.

This trial meets the recently increased demand for a systematic assessment of adverse and negative effects as well as negative experiences in IBI research (Rozental et al., 2015). Fortunately, the safety of this still unusual type of purely Internet-based guided self-help format for people with psychosis can be considered high. The number of adverse events per participant was low (0.02 events/participant) compared to CBTp in outpatient settings (e.g., 0.18 in Lincoln et al., 2012). The percentage of participants who deteriorated was also low (2% to 5%), and symptom-related negative experiences such as delusional misinterpretation of the selfhelp system were markedly rarer than intervention-related experiences such as lack of human contact (d = — 0.88). However, the contact between participants and the study team was much less frequent than in face-to-face trials, so we cannot rule out the possibility that the frequency of adverse events is underestimated in this study. Behavioral indicators of negative experiences are likely to be a useful additional source of information in future IBIs. For example, one participant spent about 76 hr logged on to the program in 8 weeks (1.4 hr per day), indicating an overuse that could have been addressed by a guide if technical means had been available for identifying and reporting such behavior during the intervention. In addition, future studies are needed that investigate whether aspects of stigmatization (e.g., social isolation, the impression that psychosis is best managed alone/virtually and should not be discussed with others) or confidentiality violations (e.g., completing the intervention in public spaces, at home with family, or on a shared computer) increase in subgroups due to using IBIs for psychosis.

Several constraints limit the explanatory scope of this trial. First, the control condition was care as usual and not an active control condition (e.g., an alternative psychological intervention with matched length). Thus, the design does not allow a test of the specificity of the effect of the intervention. Furthermore, the study was less powered than planned due to recruitment difficulties. Although small to medium-sized effects were detectable according to a sensitivity power analysis, subtler effects such as those in the secondary outcomes might have been statistically significant otherwise. Another limitation is the significant difference in duration of disorder between the iCBTp and the WL group despite randomization. An alternative explanation, which cannot be completely ruled out with data from this study, is that the iCBTp group improved more strongly due to their fewer years of psychosis and not due to the intervention itself. However, the similar number of hospitalizations in both groups (p = .792, d = — 0.06) and the essentially identical findings in the main analyses with years of psychosis introduced as covariate render this alternative explanation unlikely. Furthermore, the percentage of women in this sam-ple—approximately 58%—was higher than in RCTs with face-to-face CBT for psychosis (e.g., 44% in Lincoln et al., 2012). Because high percentages of female users are common in IBIs for other mental disorders such as depression (e.g., 67% in Karyotaki et al., 2018), the gender distribution in this sample is not surprising and might even indicate that IBIs for psychosis attract individuals who are harder to reach with face-to-face treatments. However, this unusual aspect of the sample compared to samples from

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


face-to-face CBT trials limits the generalizability of our findings to the broader population with schizophrenia-spectrum disorders who are seeking help. Other shortcomings of the trial are the noninvolvement of users in the development of the intervention, the lack of a neuropsychological assessment, and the preponderance of self-report scales as primary outcomes.

This study contributes to the young field of IBI research in psychosis, and it provides diverse directions for future research. The efficacy of IBIs as primary prevention in individuals with ultrahigh risk for psychosis or as relapse prevention in remitted individuals with psychosis is a valuable target of future research, potentially amended by (a) diagnostic interviews done by video instead of phone calls, (b) online or telephone assessments of third-party diagnostic information (e.g., family members), and (c) a long-term follow-up period within an RCT design allowing for the assessment of long-term effects. In addition, dismantling studies are needed to test which components or modules of IBIs are effective or detrimental and for whom (e.g., randomized activation or deactivation of modules). Furthermore, the development and evaluation of IBIs that focus on negative instead of positive symptoms is needed, for example as adjuncts to the pharmacological treatment of positive symptoms.

Besides paving the way for disseminating CBTp in a novel modality, the trial offers several other clinical implications. The sample was older and less educated than those in face-to-face CBTp trials (e.g., Lincoln et al., 2012), suggesting that IBIs have the capacity to reach populations that do not seek other types of psychological interventions, in some case perhaps due to fear or actual prior experiences of stigmatization in the health care system (Mestdagh & Hansen, 2014). Because the lack of personal support was the most frequently reported negative experience, embedding IBIs for psychosis in a stepped-care framework (e.g., Berger et al., 2011) with a dynamic change in the intensity of contact (e.g., phone or video calls, face-to-face sessions, and virtual chat rooms) seems useful, for instance, as a way to tailor the intervention to participants who are more difficult to engage. Furthermore, incentives such as awards for actively spending a certain amount of time in the program or completing a certain number of modules or worksheets (gamification) might help to increase adherence in future studies.

CBT-oriented IBIs for people with psychosis are a safe and feasible treatment option with an efficacy beyond care as usual, including antipsychotic medication, assuming the findings of this trial are replicated in future studies. As an alternative treatment option with low-threshold accessibility, but not as a replacement for regular face-to-face treatments, iCBTp has great potential to minimize the psychological treatment gap in this population, particularly for individuals with hallucinations.

References

Alvarez-Jimenez, M., Alcazar-Corcoles, M. A., Gonzalez-Blanch, C., Bendall, S., McGorry, P. D., & Gleeson, J. F. (2014). Online, social media and mobile technologies for psychosis treatment: A systematic review on novel user-led interventions. Schizophrenia Research, 156, 96-106. http://dx.doi.org/10.10167j.schres.2014.03.021

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5 ed.). Arlington, VA: Author.

Andersson, G. (2016). Internet-delivered psychological treatments. Annual Review of Clinical Psychology, 12, 157-179. http://dx.doi.org/10.1146/ annurev-clinpsy-021815-093006

Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018). Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. Journal of Anxiety Disorders, 55, 70-78. http://dx.doi.org/10.1016/j.janxdis.2018.01.001

Barlow, J. H., Ellard, D. R., Hainsworth, J. M., Jones, F. R., & Fisher, A. (2005). A review of self-management interventions for panic disorders, phobias and obsessive-compulsive disorders. Acta Psychiatrica Scandi-navica, 111, 272-285. http://dx.doi.org/10.1111/j.1600-0447.2005 .00499.x

Bell, M. L., Fiero, M., Horton, N. J., & Hsu, C.-H. (2014). Handling missing data in RCTs; a review of the top medical journals. BMC Medical Research Methodology, 14, 118. http://dx.doi.org/10.1186/ 1471-2288-14-118

Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014). Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophrenia Bulletin, 40, 1244-1253. http://dx.doi.org/10.1093/schbul/sbu033

Berger, T., Caspar, F., Richardson, R., Kneubuhler, B., Sutter, D., & Andersson, G. (2011). Internet-based treatment of social phobia: A randomized controlled trial comparing unguided with two types of guided self-help. Behaviour Research and Therapy, 49, 158-169. http:// dx.doi.org/10.1016/j.brat.2010.12.007

Berry, N., Lobban, F., Emsley, R., & Bucci, S. (2016). Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: A systematic review. Journal of Medical Internet Research, 18, 1-20. http://dx.doi.org/10 .2196/jmir.5250

Bighelli, I., Salanti, G., Huhn, M., Schneider-Thoma, J., Krause, M., Reitmer, C., ... Leucht, S. (2018). Psychological interventions to reduce positive symptoms in schizophrenia: Systematic review and network meta-analysis. World Psychiatry, 17, 316-329. http://dx.doi.org/10 .1002/wps.20577

Boyd Ritsher, J., Otilingam, P. G., & Grajales, M. (2003). Internalized stigma of mental illness: Psychometric properties of a new measure. Psychiatry Research, 121, 31-49. http://dx.doi.org/10.1016/j.psychres .2003.08.008

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290. http://dx.doi.org/10.1037/1040-3590.6.4.284

Cuijpers, P., & Schuurmans, J. (2007). Self-help interventions for anxiety disorders: An overview. Current Psychiatry Reports, 9, 284-290. http:// dx.doi.org/10.1007/s11920-007-0034-6

Ebert, D. D., Donkin, L., Andersson, G., Andrews, G., & Cuijpers, P. (2016). Does Internet-based guided-self-help for depression cause harm? An individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychological Medicine, 46, 2679-2693. http://dx.doi.org/10.1017/S0033291716001562

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G’Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. http:// dx.doi.org/10.3758/BF03193146

Freeman, D., & Garety, P. (2014). Advances in understanding and treating persecutory delusions: A review. Social Psychiatry and Psychiatric Epidemiology, 49, 1179-1189. http://dx.doi.org/10.1007/s00127-014-0928-7

Freeman, D., Garety, P. A., Bebbington, P. E., Smith, B., Rollinson, R., Fowler, D., . . . Dunn, G. (2005). Psychological investigation of the structure of paranoia in a non-clinical population. British Journal of Psychiatry, 186, 427-435. http://dx.doi.org/10.1192/bjp.186.5.427

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


Garety, P. A., & Freeman, D. (2013). The past and future of delusions research: From the inexplicable to the treatable. British Journal of Psychiatry, 203, 327-333. http://dx.doi.org/10.1192/bjp.bp.113.126953

Gottlieb, J. D., Romeo, K. H., Penn, D. L., Mueser, K. T., & Chiko, B. P. (2013). Web-based cognitive-behavioral therapy for auditory hallucinations in persons with psychosis: A pilot study. Schizophrenia research, 145, 82-87. http://dx.doi.org/10.1016/j.schres.2013.01.002

Grosse Holtforth, M. (2008). Avoidance motivation in psychological problems and psychotherapy. Psychotherapy Research, 18, 147-159. http:// dx.doi.org/10.1080/10503300701765849

Haddock, G., Eisner, E., Boone, C., Davies, G., Coogan, C., & Barrowclough, C. (2014). An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia. Journal of Mental Health, 23, 162-165. http://dx.doi.org/10.3109/09638237 .2013.869571

Hazell, C. M., Hayward, M., Cavanagh, K., Jones, A.-M., & Strauss, C. (2018). Guided self-help cognitive-behaviour intervention for VoicEs (GiVE): Results from a pilot randomised controlled trial in a transdiagnostic sample. Schizophrenia Research, 195, 441-447. http://dx.doi.org/ 10.1016/j.schres.2017.10.004

Irfan, M., Muzaffar, S., Kingdon, D., Rathod, S., & Naeem, F. (2019). Psychotherapy for schizophrenia and bipolar disorder. In D. J. Stein,

J. K. Bass, & S. G. Hofmann (Eds.), Global Mental Health and Psychotherapy (pp. 223-239). Amsterdam, the Netherlands: Elsevier. http:// dx.doi.org/10.1016/B978-0-12-814932-4.00010-0

Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352. http://dx.doi.org/ 10.1016/S0005-7894(84)80002-7

Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., & Laws,

K. R. (2014). Cognitive-behavioural therapy for the symptoms of schizophrenia: Systematic review and meta-analysis with examination of potential bias. British Journal of Psychiatry, 204, 20-29. http://dx.doi.org/ 10.1192/bjp.bp.112.116285

Jin, H., & Mosweu, I. (2017). The societal cost of schizophrenia: A systematic review. PharmacoEconomics, 35, 25-42. http://dx.doi.org/ 10.1007/s40273-016-0444-6

Karyotaki, E., Kemmeren, L., Riper, H., Twisk, J., & Cuijpers, P. (2018). Is self-guided internet-based cognitive behavioural therapy (iCBT) harmful? An individual participant data meta-analysis. Psychological Medicine, 48, 2456-2466. http://dx.doi.org/10.1017/S003329 1718000648

Karyotaki, E., Kleiboer, A., Smit, F., Turner, D. T., & Cuijpers, P. (2015). Predictors of treatment dropout in self-guided web-based interventions for depression: An ‘individual patient data' meta-analysis. Psychological Medicine, 45, 2717-2726. http://dx.doi.org/10.1017/S0033 291715000665

Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13, 261-276. http://dx.doi.org/10.1093/schbul/13.2.261

Kim, S. W., Lee, G. Y., Yu, H. Y., Jung, E. I., Lee, J. Y., Kim, S. Y.,... Yoon, J. S. (2018). Development and feasibility of smartphone application for cognitive-behavioural case management of individuals with early psychosis. Early Intervention in Psychiatry, 12, 1087-1093. http:// dx.doi.org/10.1111/eip.12418

Kiran, C., & Chaudhury, S. (2016). Prevalence of comorbid anxiety disorders in schizophrenia. Industrial Psychiatry Journal, 25, 35- 40.

Krakvik, B., Grawe, R. W., Hagen, R., & Stiles, T. C. (2013). Cognitive behaviour therapy for psychotic symptoms: A randomized controlled effectiveness trial. Behavioural and Cognitive Psychotherapy, 41, 511524. http://dx.doi.org/10.1017/S1352465813000258

Launay, G., & Slade, P. (1981). The measurement of hallucinatory predisposition in male and female prisoners. Personality and Individual Differences, 2, 221-234. http://dx.doi.org/10.1016/0191-8869(81)90027-1

Lecrubier, Y., Sheehan, D., Weiller, E., Amorim, P., Bonora, I., Harnett Sheehan, K., . . . Dunbar, G. (1997). The mini international neuropsychiatric interview (MINI). A short diagnostic structured interview: Reliability and validity according to the CIDI. European Psychiatry, 12, 224-231. http://dx.doi.org/10.1016/S0924-9338(97)83296-8

Lincoln, T. M., Keller, E., & Rief, W. (2009). Die erfassung von wahn und halluzinationen in der normalbevolkerung: Deutsche adaptationen des Peters et al. delusions inventory (PDI) und der Launay Slade hallucination scale (LSHS-R). Diagnostica, 55, 29-40. http://dx.doi.org/10.1026/ 0012-1924.55.1.29

Lincoln, T., Pedersen, A., Hahlweg, K., Wiedl, K.-H., & Frantz, I. (2019). Evidenzbasierte Leitlinie zur Psychotherapie von Schizophrenie und anderen psychotischen Storungen [Evidence-based guidelines for the psychological therapy of schizophrenia and other psychotic disorders]. Gottingen, Germany: Hogrefe. http://dx.doi.org/10.1026/02883-000

Lincoln, T. M., Ziegler, M., Lullmann, E., Muller, M. J., & Rief, W. (2010). Can delusions be self-assessed? Concordance between self- and observer-rated delusions in schizophrenia. Psychiatry Research, 178, 249-254. http://dx.doi.org/10.1016/j.psychres.2009.04.019

Lincoln, T. M., Ziegler, M., Mehl, S., Kesting, M.-L., Lullmann, E., Westermann, S., & Rief, W. (2012). Moving from efficacy to effectiveness in cognitive behavioral therapy for psychosis: A randomized clinical practice trial. Journal of Consulting and Clinical Psychology, 80, 674-686. http://dx.doi.org/10.1037/a0028665

Ludtke, T., Platow-Kohlschein, H., Ruegg, N., Berger, T., Moritz, S., & Westermann, S. (2020). Mindfulness mediates the effect of a psychological online intervention for psychosis on self-reported hallucinations: A secondary analysis of voice hearers from the EviBaS trial. Frontiers in Psychiatry, 11, 228. http://dx.doi.org/10.3389/fpsyt.2020.00228

Ludtke, T., Westermann, S., Pult, L. K., Schneider, B. C., Pfuhl, G., & Moritz, S. (2018). Evaluation of a brief unguided psychological online intervention for depression: A controlled trial including exploratory moderator analyses. Internet Interventions, 13, 73-81. http://dx.doi.org/ 10.1016/j.invent.2018.06.004

Mestdagh, A., & Hansen, B. (2014). Stigma in patients with schizophrenia receiving community mental health care: A review of qualitative studies. Social Ps\chiatr\ and Psychiatric Epidemiology, 49, 79-87. http://dx .doi.org/10.1007/s00127-013-0729-4

Moritz, S., Andreou, C., Schneider, B. C., Wittekind, C. E., Menon, M., Balzan, R. P., & Woodward, T. S. (2014). Sowing the seeds of doubt: A narrative review on metacognitive training in schizophrenia. Clinical Psychology Review, 34, 358-366. http://dx.doi.org/10.1016/j.cpr.2014 .04.004

Moritz, S., Goritz, A. S., Balzan, R. P., Gaweda, L., Kulagin, S. C., & Andreou, C. (2017). A new paradigm to measure probabilistic reasoning and a possible answer to the question why psychosis-prone individuals jump to conclusions. Journal of Abnormal Psychology, 126, 406-415. http://dx.doi.org/10.1037/abn0000262

Moritz, S., Klein, J. P., Desler, T., Lill, H., Gallinat, J., & Schneider, B. C. (2017). Neurocognitive deficits in schizophrenia. Are we making mountains out of molehills? Psychological Medicine, 47, 2602-2612. http:// dx.doi.org/10.1037/abn0000262

Moritz, S., Scheunemann, J., Ludtke, T., Westermann, S., Pfuhl, G., Balzan, R. P., & Andreou, C. (in press). Prolonged rather than hasty decision-making in schizophrenia using the box task. Must we rethink the jumping to conclusions account of paranoia? Schizophrenia Research. http://dx.doi.org/10.1016/j.schres.2020.05.056

Moritz, S., Schroder, J., Klein, J. P., Lincoln, T. M., Andreou, C., Fischer, A., & Arlt, S. (2016). Effects of online intervention for depression on mood and positive symptoms in schizophrenia. Schizophrenia Research, 175(1-3), 216-222. http://dx.doi.org/10.1016/j.schres.2016.04.033

Moritz, S., Veckenstedt, R., Andreou, C., Bohn, F., Hottenrott, B., Leighton, L., .. . Roesch-Ely, D. (2014). Sustained and “sleeper” effects of group metacognitive training for schizophrenia: A randomized clin-

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.


ical trial. Journal of the American Medical Association Psychiatry, 71, 1103-1111. http://dx.doi.org/10.1001/jamapsychiatry.2014.1038

Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364-386. http:// dx.doi.org/10.1177/1094428106291059

Morrison, A. P., Turkington, D., Pyle, M., Spencer, H., Brabban, A., Dunn, G., . . . Hutton, P. (2014). Cognitive therapy for people with schizophrenia spectrum disorders not taking antipsychotic drugs: A singleblind randomised controlled trial. Lancet, 383, 1395-1403. http://dx.doi .org/10.1016/S0140-6736(13)62246-1

National Institute for Health and Care Excellence. (2014). Psychosis and schizophrenia in adults treatment and management. Updated edition. Clinical guidelines no. 178. London: National Institute for Health and Care Excellence.

Pinto, A., Gigantesco, A., Morosini, P., & La Pia, S. (2007). Development, reliability and validity of a self-administered questionnaire on subjective opinion about delusions and voices. Psychopathology, 40, 312-320. http://dx.doi.org/10.1159/000105529

Rotondi, A. J., Anderson, C. M., Haas, G. L., Eack, S. M., Spring, M. B., Ganguli, R., . . . Rosenstock, J. (2010). Web-based psychoeducational intervention for persons with schizophrenia and their supporters: One-year outcomes. Psychiatric services, 61, 1099-1105.

Rotondi, A. J., Eack, S. M., Hanusa, B. H., Spring, M. B., & Haas, G. L. (2015). Critical design elements of e-health applications for users with severe mental illness: Singular focus, simple architecture, prominent contents, explicit navigation, and inclusive hyperlinks. Schizophrenia Bulletin, 41, 440-448. http://dx.doi.org/10.1093/schbul/sbt194

Rozental, A., Boettcher, J., Andersson, G., Schmidt, B., & Carlbring, P. (2015). Negative effects of internet interventions: A qualitative content analysis of patients' experiences with treatments delivered online. Cognitive Behaviour Therapy, 44, 223-236. http://dx.doi.org/10.1080/ 16506073.2015.1008033

Ruegg, N., Moritz, S., Berger, T., Ludtke, T., & Westermann, S. (2018). An internet-based intervention for people with psychosis (EviBaS): Study protocol for a randomized controlled trial. BMC Psychiatry, 18, 102. http://dx.doi.org/10.1186/s12888-018-1644-8

Ruegg, N., Moritz, S., & Westermann, S. (2018). Metacognitive training online: A pilot study of an internet-based intervention for people with Schizophrenia. Zeitschrift fur Neuropsychologie, 29, 35-37. http://dx .doi.org/10.1024/1016-264X/a000213

Schlosser, D. A., Campellone, T. R., Truong, B., Etter, K., Vergani, S., Komaiko, K., & Vinogradov, S. (2018). Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophrenia Bulletin, 44, 1010-1020. http://dx.doi.org/ 10.1093/schbul/sby078

Schmidt, J., & Wittmann, W. W. (2002). ZUF-8: Fragebogen zur Messung der Patientenzufriedenheit [ZUF-8: Client Satisfaction Questionnaire]. Gottingen, Germany: Hogrefe.

Schroder, J., Sautier, L., Kriston, L., Berger, T., Meyer, B., Spath, C., . . . Moritz, S. (2015). Development of a questionnaire measuring attitudes towards psychological online interventions-the APOI. Journal of Affective Disorders, 187, 136-141. http://dx.doi.org/10.1016/jjad.2015.08 .044

Scott, A. J., Webb, T. L., & Rowse, G. (2015). Self-help interventions for psychosis: A meta-analysis. Clinical Psychology Review, 39, 96 -112. http://dx.doi.org/10.1016/j.cpr.2015.05.002

Stolz, T., Schulz, A., Krieger, T., Vincent, A., Urech, A., Moser, C., . . . Berger, T. (2018). A mobile app for social anxiety disorder: A three-arm randomized controlled trial comparing mobile and PC-based guided self-help interventions. Journal of Consulting and Clinical Psychology, 86, 493-504. http://dx.doi.org/10.1037/ccp0000301

Tarrier, N., Yusupoff, L., Kinney, C., McCarthy, E., Gledhill, A., Haddock, G., & Morris, J. (1998). Randomised controlled trial of intensive cognitive behaviour therapy for patients with chronic schizophrenia. British Medical Journal, 317, 303-307. http://dx.doi.org/10.1016/j.schres.2006 .03.021

Titov, N., Dear, B., Nielssen, O., Staples, L., Hadjistavropoulos, H., Nugent, M., . . . Kaldo, V. (2018). ICBT in routine care: A descriptive analysis of successful clinics in five countries. Internet Interventions, 13, 108-115. http://dx.doi.org/10.1016j.invent.2018.07.006

van der Gaag, M., Hoffman, T., Remijsen, M., Hijman, R., . . . Wiersma, D. (2006). The five-factor model of the positive and negative syndrome scale II: A ten-fold cross-validation of a revised model. Schizophrenia Research, 85, 280-287. http://dx.doi.org/10.1016j.schres.2006.03.021

van der Krieke, L., Wunderink, L., Emerencia, A. C., de Jonge, P., & Sytema, S. (2014). E-mental health self-management for psychotic disorders: State of the art and future perspectives. Psychiatric Services, 65, 33-49. http://dx.doi.org/10.1176/appi.ps.201300050

Velthorst, E., Koeter, M., van der Gaag, M., Nieman, D. H., Fett, A. K., Smit, F., . . . de Haan, L. (2015). Adapted cognitive-behavioural therapy required for targeting negative symptoms in schizophrenia: Metaanalysis and meta-regression. Psychological Medicine, 45, 453- 465. http://dx.doi.org/10.1017/S0033291714001147

Westermann, S., Cavelti, M., Heibach, E., & Caspar, F. (2015). Motive-oriented therapeutic relationship building for patients diagnosed with schizophrenia. Frontiers in Psychology, 6, 1294. http://dx.doi.org/10 .3389/fpsyg.2015.01294

WHOQOL Group. (1998). Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological Medicine, 28, 551-558. http://dx.doi.org/10.1017/S0033291798006667

Wind, T. R., Rijkeboer, M., Andersson, G., & Riper, H. (2020). The COVID-19 pandemic: The ‘black swan' for mental health care and a turning point for e-health. Internet Interventions, 20, 100317. http://dx .doi.org/10.1016/j.invent.2020.100317

Zagorscak, P., Heinrich, M., Sommer, D., Wagner, B., & Knaevelsrud, C. (2018). Benefits of individualized feedback in internet-based interventions for depression: A randomized controlled trial. Psychotherapy and Psychosomatics, 87, 32-45. http://dx.doi.org/10.1159/000481515

Received February 27, 2020

Revision received June 18, 2020

Accepted June 18, 2020 ■


Checkfor updates


£ OPEN ACCESS

Citation: Xu D(Roman), Xiao S, He H, Caine ED, Gloyd S, Simoni J, etal. (2019) Lay health supporters aided by mobile text messaging to improve adherence, symptoms, and functioning among people with schizophrenia in a resourcepoor community in rural China (LEAN): A randomized controlled trial. PLoS Med 16(4): e1002785. https://doi.org/10.1371/journal. pmed.1002785

Academic Editor: Vikram Patel, Harvard Medical School, UNITED STATES

Received: October 8,2018

Accepted: March 20,2019

Published: April 23,2019

Copyright: © 2019 Xu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability Statement: The data underlying the results presented in the study are available from a data repository at Research Gate (https:// www.researchgate.net/publication/328075324 LEAN Phase 1 Data) DOI: 10.13140/RG.2.2. 18638.51527

Funding: The project received grant support from the China Medical Board (https://


RESEARCH ARTICLE

Lay health supporters aided by mobile text messaging to improve adherence, symptoms, and functioning among people with schizophrenia in a resource-poor community in rural China (LEAN): A randomized controlled trial

Dong (Roman) Xu|D1, Shuiyuan Xiao2, Hua Hen?3, Eric D. Caine4, Stephen Gloyd5, Jane Simoni6, James P. Hughes7, Juan Nie1, Meijuan Lin2, Wenjun He1, Yeqing Yuan8, Wenjie Gong®2*

1 Sun Yat-sen Global Health Institute, School of Public Health and Institute of National Governance, Sun Yat-sen University, Guangzhou, Guangdong, China, 2 Xiangya School of Public Health, Central South University, Changsha, Hunan, China, 3 Department of Epidemiology, School of Public Health and Tropical Medicine, Tulane University, New Orleans, Louisiana, United States of America, 4 Department of Psychiatry, University of Rochester Medical Center, Rochester, New York, United States of America, 5 Department of Global Health, University of Washington, Seattle, Washington, United States of America, 6 Department of Psychology, University of Washington, Seattle, Washington, United States of America, 7 Department of Biostatistics, University of Washington, Seattle, Washington, United States of America, 8 Silver School of Social Work, New York University, New York, New York, United States of America

* gongwenjie@csu.edu.cn

Abstract

Background

Schizophrenia is a leading cause of disability, and a shift from facility- to community-based care has been proposed to meet the resource challenges of mental healthcare in low- and middle-income countries. We hypothesized that the addition of mobile texting would improve schizophrenia care in a resource-poor community setting compared with a communitybased free-medicine program alone.

Methods and findings

In this 2-arm randomized controlled trial, 278 community-dwelling villagers (patient participants) were randomly selected from people with schizophrenia from 9 townships of Hunan, China, and were randomized 1:1 into 2 groups. The program participants were recruited between May 1,2015, and August 31,2015, and the intervention and follow-up took place between December 15,2015, and July 1,2016. Baseline characteristics of the 2 groups were similar. The patients were on average 46 years of age, had 7 years of education, had a duration of schizophrenia of 18 years with minimal to mild symptoms and nearly one-fifth loss of functioning, and were mostly living with family (95%) and had low incomes. Both the intervention and the control groups received a nationwide community-based mental health

chinamedicalboard.org/; grant number 12-114, WG, PI) and NIH (https://fogartycenter.org; research training grant #R25 TW009345, DX, Fogarty fellowship). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: AMD, adjusted mean difference; BARS, Brief Adherence Rating Scale; CGI, Clinical Global Impression; DAI-10, Drug Attitude Inventory-10; GEE, generalized estimating equation; HBM, health belief model; LMICs, low-and middle-income countries; mHealth, mobile health; mhGAP, Mental Health Gap Action Programme; NNT, number needed to treat; RR, relative risk; THC, township health center; WHODAS, WHO Disability Assessment Schedule 2.0.


program that provided free antipsychotic medications. The patient participants in the intervention group also received LEAN (Lay health supporters, E-platform, Award, and iNtegration), a program that featured recruitment of a lay health supporter and text messages for medication reminders, health education, monitoring of early signs of relapses, and facilitated linkage to primary healthcare. The primary outcome was medication adherence (proportion of dosages taken) assessed by 2 unannounced home-based pill counts 30 days apart at the 6-month endpoint. The secondary and other outcomes included patient symptoms, functioning, relapses, re-hospitalizations, death for any reason, wandering away without notifying anyone, violence against others, damaging goods, and suicide. Intent-to-treat analysis was used. Missing data were handled with multiple imputations. In total, 271 out of 278 patient participants were successfully followed up for outcome assessment. Medication adherence was 0.48 in the control group and 0.61 in the intervention group (adjusted mean difference [AMD] 0.12 [95% CI 0.03 to 0.22]; p = 0.013; effect size 0.38). Among secondary and other outcomes we noted substantial reduction in the risk of relapse (26 [21.7%] of 120 interventional participants versus 40 [34.2%] of 117 controls; relative risk 0.63 [95% CI 0.42 to 0.97]; number needed to treat [NNT] 8.0) and re-hospitalization (9 [7.3%] of 123 interventional participants versus 25 [20.5%] of 122 controls; relative risk 0.36 [95% CI 0.17 to 0.73]; NNT 7.6). The program showed no statistical difference in all other outcomes. During the course of the program, 2 participants in the intervention group and 1 in the control group died. The limitations of the study include its lack of a full economic analysis, lack of individual tailoring of the text messages, the relatively short 6-month follow-up, and the generalizability constraint of the Chinese context.

Conclusions

The addition of texting to patients and their lay health supporters in a resource-poor community setting was more effective than a free-medicine program alone in improving medication adherence and reducing relapses and re-hospitalizations. Future studies may test the effectiveness of customization of the texting to individual patients.

Trial registration

Chinese Clinical Trial Registry ChiCTR-ICR-15006053.

Author summary

Why was this study done?

What did the researchers do and find?

What do these findings mean?

Introduction

Schizophrenia is a leading cause of disability, with a global prevalence of 0.4% [1] and contributing to 1.7% of total years lived with disability [2]. Schizophrenia also leads to a high economic burden [3] and the violation of human rights due to the stigmatization of the illness [4] and other causes. The WHO's Mental Health Gap Action Programme (mhGAP) has identified schizophrenia as the top priority for global action, recommending treatment with antipsychotic medicines and psychosocial care [5]. However, in low- and middle-income countries (LMICs) the treatment gap remains high [6-8], and even when treatment is available, adherence to antipsychotics is low compared with adherence to drugs for other diseases because of patients' lack of insight into their illness, their forgetfulness due to cognitive impairment, and side effects that are often associated with schizophrenia [9,10]. In LMICs, with limited mental health facilities and the healthcare workforce concentrated in large urban centers, the scarcity, inequity, and inefficiency of mental health resources present challenges [11,12]. As a result, there is a broad consensus for collaborative stepped care that emphasizes community- and family-based treatment, task sharing among human resources, and integrating mental health into existing primary healthcare [5,7,13].

In community- and family-based healthcare, mobile health (mHealth) has gained increasing traction [14-16]. With the proliferation of mHealth studies, there is a tendency to pursue “advanced” and often complicated mHealth solutions instead of simple but reliable methods that may work better among poorly educated populations in low-resource settings. Short message service (SMS) text messaging, or texting, as a simple and technologically reliable method, has been shown to be particularly useful in resource-poor settings due to its wide availability, reliability, ease of use, and relatively low cost [15,17]. Since the first publication of studies involving texting for health in 2002 [18,19], text messaging has been found to benefit diabetes self-management, weight loss, physical activity, smoking cessation, and medication adherence to antiretroviral therapy [17,20]. For people with serious mental disorders such as schizophrenia, texting has been used in 4 areas of application: reminders for medication and clinical appointments [21,22], information dissemination, supportive messaging, and self-monitoring procedures [15]. However, despite the recent proliferation of text messaging, there has been no clear evidence that technology-based prompts improve treatment adherence, symptoms, or functioning in people with schizophrenia [23,24]. Most studies to date have been small pilot studies that focused on feasibility rather than health outcomes [25,26], were primarily conducted in high-income countries [15,25], did not include informal caregivers (who often play important roles in schizophrenia management) [27], and often served as a stand-alone intervention not integrated with the health system [28]. Also, these studies paid insufficient attention to user evaluation and appreciation of the program [20] and to establishing a theoretical basis or working mechanism for the mobile intervention [29].

The LEAN intervention was intended to address some of the insufficiencies of the previous trials mentioned above and to have broad applicability for resource-poor settings in LMICs. This study was conducted as a pragmatic trial to test our primary hypothesis that lay health supporters (family members of the patients or community volunteers) aided by a simple texting system would increase patient adherence to antipsychotic medications, and improve symptoms and functioning in a community-based cohort of patients.

Methods

Study design and participants

We designed a 2-arm individually based randomized controlled trial. Details of the study design, methods, and analysis plan have previously been published as a study protocol [30]. The trial was prospectively registered in the Chinese Clinical Trial Registry (ChiCTR-ICR-15006053). Individual rather than cluster design was used as the likelihood of contamination or spillover effects of the intervention was considered minimal due to the private nature of our texting and tendency of patients and their families in rural China not to discuss mental conditions publicly. The study recruited and followed-up the patients between May 1, 2015, and July 1, 2016, in 9 rural townships (population 356,900) of Liuyang Municipality, Hunan Province, in central China.

We applied minimal inclusion and exclusion criteria [30]. The inclusion criteria were (1) being community-dwelling, (2) being an enrollee of the National Continuing Management and Intervention Program for Psychoses, known as the “686 Program,” (3) having a primary diagnosis of schizophrenia, (3) being on oral psychotropic medications, and (4) physically residing in 1 of 9 rural townships of Liuyang Municipality. People were excluded if they (1) were hospitalized due to schizophrenia at the time of recruitment (our intervention required sustained community residence), (2) had missed the most recent 3 consecutive past drug refills (in this case, they had de facto dropped out of the 686 Program), or (3) were physically incapable of using voice and text messaging (hearing and/or vision impairment prevented the use of our

intervention). Trial participants were selected by simple random sampling from the 686 Program registry, which included almost all known villagers diagnosed with schizophrenia in Liuyang. The diagnosis of schizophrenia in the 686 Program used ICD-10 [31]. Trained interviewers obtained written informed consent from both the patient and the lay health supporters.

Procedures

The development of the intervention, LEAN (Lay health supporters, E-platform, Award, and iNtegration), was guided by theory, empirical evidence, and our trial and error. In early 2015, challenged by low medication adherence among 686 Program enrollees, we piloted a program in rural Liuyang that tasked “village doctors” (paraprofessionals with rudimentary medical training) to directly deliver and monitor medication ingestion in patient homes [32]. However, the village doctors were already overburdened and had neither the time nor the incentive to take on more work. After rounds of consultation with policy-makers, clinicians, and patients and their families, we reached a consensus that only a low-cost, low-burden, easy-to-imple-ment, and easy-to-use intervention was acceptable. Subsequently, following an adapted health belief model (HBM) [33], we selected individual components of LEAN from empirical literature on “task sharing,” medication adherence, and mHealth to improve patient adherence to medication. According to the HBM, people with schizophrenia may adhere to medications if they are properly “cued” to action (e.g., by prompts from the LEAN text messages and the lay health supporters to take medications) after weighing the net benefits of the medication against the perceived threat of schizophrenia (e.g., LEAN texted messages to improve patients' understanding of the benefits of medication and the consequences of not controlling schizophrenia, and to enhance their handling of relapse and side effects) (details elsewhere [30]).

The acronym LEAN summarizes the 4 program components: Lay health supporters, E-plat-form, Award, and iNtegration. A lay health supporter was selected from the patient family or the community, who followed phone-texted instructions to perform simple tasks: supervising patient medication, monitoring side effects and relapse, and facilitating urgent care. We approached the people in the following order in identifying the lay health supporters: (1) the main family caregivers registered on the 686 Program registry, (2) family members accompanying the patients to collect medication refills, (3) family members nominated by the patients, and (4) community volunteers nominated by the mental health administrator with the agreement from the patients. The e-platform (an existing commercial telemarketing system) texted 2 daily messages to both the patients and their lay health supporters: a message at 9:00 AM with educational information on schizophrenia and a reminder at 7:00 PM to take medicine (see S2 Appendix for sample messages). All messages were phrased to be caring, polite, and personal as those characteristics were patient-preferred. To reduce user fatigue, the medication reminder was embedded in a message about local weather and news (e.g., “Good evening. Tomorrow: Sunny, 23 degrees. Go out and enjoy the Temple Fairs in ABC village. Please text 1 after taking medicine.”). We also sent occasional messages with a 14-item checklist for early signs of relapse [34] and medication side effects. The lay health supporter was expected to text back “1” if any item was checked, to which a project coordinator would follow up with a phone call. A group of master's and doctoral students in public health and medicine were tasked to produce the messages, mainly adapting contents from evidence-based sources. A senior psychiatrist reviewed and approved messages for use. Every week our project coordinator prepared a texting report based on the messaging server data that included a list of families who texted back more frequently than the past month to confirm the taking of medication. The mental health administrators used the lists to award this improvement with a token gift such as a bar of soap and a congratulatory text message on a monthly basis. Finally, text messaging

also served as a communication tool that integrated the efforts of lay health supporters into the existing health system—one example of this being an arrangement between village doctors, the project coordinator, and patients' psychiatrists that if signs of relapse were detected, village doctors were texted to assess severity and the project coordinator then scheduled an appointment with the psychiatrist and texted appointment details to the patient's family.

The patients in the intervention group and their lay health supporters received training in preparation for LEAN. The lay health supporters received a brief introduction of their responsibilities and roles at the time of recruitment, a demonstration on phone use (for those who had not had a phone and were given a free phone), and continued training through our text messages on how to provide patient care and seek professional help. The patients were evaluated for their ability to use 3 basic phone functions: turning the phone on/off, charging the phone, and reading/returning text messages. The patients in need of training received up to 3 sessions (20 minutes per session) of hands-on demonstration on how to use their phones (details elsewhere [35]).

After a 3-month pilot of LEAN, we made several design and implementation changes. Among the major changes, we decided to text medication reminders to both the lay health supporters and the patients rather than the patients alone, and to use an existing telemarketing platform rather than the one we developed on our own (the existing platform was less expensive, easier to implement, and technologically more sophisticated).

While the intervention group received the 686 Program plus LEAN, the control group received the 686 Program alone. The 686 Program is a national public program covering 5.4 million people with psychosis in China, three-fourths of whom are people with schizophrenia [36]. Despite some local variations, the basic structure of the 686 Program remains the same across the country. In Liuyang, a psychiatrist served as the full-time program director, supported by 2 other psychiatrists and several staff members working part-time (the psychiatrists were internists who converted their roles through on-the-job training). The psychiatrists together with several staff members traveled every 2 months to each township health center (THC) to provide patient consultations and free medication. The township mental health administrators, supervised by both the psychiatrists and the local THCs, coordinated the work of the village doctors to provide regular services, which included yearly physical exams, assessment of risk level, >4 home visits throughout the year, health education, and urgent care.

Randomization and masking

The 686 Program registry included as high a proportion of villagers with schizophrenia as possible. We first selected 400 names from the registry with simple random sampling. Then we applied inclusion and exclusion criteria to this group and recruited those eligible after obtaining their informed consent. After the patients were recruited, a statistician not otherwise associated with the project allocated participants (1:1) by simple randomization to receive either the 686 Program plus LEAN (intervention group) or the 686 Program alone (control group). It was not possible to blind the program participants to allocation. However, outcome assessors were blind to group assignment and were physically separated from the program implementation team, which operated LEAN and monitored user experiences. The psychiatrists were also blind to the allocation status of participants. If unmasking occurred inadvertently during assessment, this was to be reported immediately, and a make-up assessment scheduled with separate assessors.

Outcomes

Our published trial protocol provided details on measurements [30]. In brief, the primary outcome was a score of adherence to antipsychotic medications—the proportion of dosages taken

over the past 30 days. The measurement was primarily based on unannounced home-based pill counts [37], and we followed the consensus guideline of including both objective and subjective measures when assessing adherence [38-40]. The pill counts were unannounced to minimize a potential “Hawthorne effect” [41]. To obtain 1 measurement of adherence, our evaluators made 2 home-based pill counts 30 days apart at the fifth month and the sixth month, and the difference between the 2 counts was considered the number of pills taken. We developed specific procedures to handle multiple bottles, discarded pills, and additionally acquired pills in between the 2 counts. The number of pills prescribed for that period was obtained from the 686 Program medication prescribing system. Adherence was calculated as (number of first count - number of second count + number of additional pills obtained - number of pills discarded) (number of pills prescribed). The counts were unannounced in the sense that although patients knew we would assess their adherence, they did not know on which home visit we would count pills, as village doctors scheduled those visits with the 686 Program routine visits whenever possible. To supplement pill counts, we also measured adherence through adherence rating scales (Brief Adherence Rating Scale [BARS] [42] and the Drug Attitude Inventory-10 [DAI-10] [43]) and medication refill records. We did not use electronic medicine caps to measure adherence as they provide a form of adherence intervention in themselves and do not improve the accuracy of measurement above that of unannounced pill counts. Due to resource constraints, we used refill records, DAI-10, and BARS instead of pill counts for the measurement of the baseline adherence. The rationale, validity, and details of our methods to measure adherence to antipsychotics in LEAN were published and are publicly available elsewhere [44]. Secondary outcomes were patient symptoms measured by the Clinical Global Impression (CGI) for schizophrenia (which includes 2 scales: CGI-Severity and CGI-Improvement) [45] and patient functioning measured by the WHO Disability Assessment Schedule 2.0 (WHODAS) [46]. The 686 Program psychiatrists administered CGI on patients, and the trained public health master's and doctoral students assessed adherence and functioning. Outcomes were assessed at baseline and 6 months. The 686 Program registry provided additional data on patient attendance for clinical appointments, relapses (defined as an overall and marked increase in symptoms assessed by the health professionals through interviewing patients and family members according to the 686 Program protocol), re-hospitalization due to schizophrenia, and incidence of death for any reason, wandering, violence against others, damaging goods, and suicide. We also captured information on program cost and user experiences from various program operation channels and patient surveys. At baseline, we used the Glasgow Antipsychotic Side-effect Scale (GASS) to assess patient-reported side effects of antipsychotics, but we did not assess side effects as an outcome during follow-up due to a program administrative mishap. All data were double-entered into and managed by REDCap, a webbased secured data management tool [47].

Statistical analysis

Using adherence data based on the clinician impression from the 686 Program management system, we determined that an increase of medication adherence from 0.72 to 0.85 (SD 0.33) would be a minimally important difference after consultation with 686 Program policy-makers. Following a standard procedure for a hypothesis of equal population means based on test, we calculated that a total sample size of 258 participants (129 per group) would have 85% power to detect an increase of medication adherence from 0.72 to 0.85, assuming a 5% type I error and 10% attrition. All analyses including the subgroup analyses were conducted as prespecified in the protocol [30]. Statistical package R was used.

We first performed a descriptive analysis of the data: sociodemographic information, key covariates, and outcomes at baseline were compared between the intervention and control groups to assess the randomization and participant characteristics. For the analysis of the program effect, intent-to-treat analysis was used for all participants. Missing outcomes were imputed based on demographic information and some other outcomes using the R package MICE [48], and 10 sets of data for each outcome were imputed (S1 Appendix). We used a semi-parametric generalized estimating equation (GEE) model to analyze program effect on adherence, symptoms, and functioning (i.e., medication adherence scores, WHODAS scores, and CGI-Severity scores, respectively) at the endpoint. We used GEE instead of analysis of covariance (ANCOVA) as initially proposed on our protocol due to a potential violation of the normal distribution assumption required for ANCOVA. Adherence analysis was adjusted for baseline adherence, WHODAS, and CGI scores; substance use; medication side effects; and family supervision, all of which are empirically suggested strong baseline predictors of adherence and pre-specified in our protocol. WHODAS and CGI analyses were adjusted for baseline WHODAS and CGI scores, respectively. We used the same GEE models for the analyses of 2 subgroups identified by medication refill adherence over the past year at baseline (people who collected all 6 refills were considered adherent, and those who missed any of the 6 refills were considered nonadherent) and functioning (cutoff at 0.22). To enable cross-study comparison, we calculated the program effect size as Cohen's d [49].

Ethical approval

The study protocol was approved by the institutional review boards of the University of Washington (49464 G) in Seattle, Washington, US, and Central South University (CTXY-150002-6) in Hunan, China. All patients and lay health supporters in this study gave their written informed consent before taking part in the study.

Results

Participant profile

Fig 1 shows the trial profile. Due to a higher rate of signing up to the program than expected, we recruited 278 patients out of the 400 candidates randomly selected from the 686 Program registry, slightly more than what we had planned. These 278 enrolled patients were randomized 1:1 into the intervention and control groups. Among the 400 candidates we approached to recruit, 12 people refused the enrollment, among whom most did not provide specific reasons for refusal; 56 (14%) did not satisfy our inclusion/exclusion criteria (e.g., some were not eligible as their primary diagnosis was epilepsy or another mental condition rather than schizophrenia); and 54 were not successfully contacted for consent due to various reasons (e.g., wrong contact information in the registry or not available/present at the time of our recruitment visits). The recruitment of the program participants occurred between May 1, 2015, and August 31, 2015, and the program was piloted from September 1 to November 30, 2015. The official intervention and follow-up took place between December 15,2015, and July 1, 2016. Six participants (4%) from the intervention group and 1 (0.7%) from the control group were lost to follow-up at the 6-month assessment. No outcome interviews were unmasked throughout the trial. The sociodemographic and clinical profiles were comparable between the intervention and control groups at baseline (Table 1) (see S3 Appendix for free medications dispensed by the 686 Program). The patients were on average 46 years of age, had 7 years of education, had a duration of schizophrenia of 18 years with minimal to mild symptoms and nearly one-fifth loss of functioning, and mostly lived with family (95%) and had low incomes. Each patient in the intervention group was successfully assigned a lay health

supporter: 80.6% of the lay health supporters were family members (mostly spouses and parents), and the remaining were community volunteers (Table 1).

Process indicators

We collected a range of process indicators related to training, program implementation, user experiences, and content of family care. For training, we were able to assess 103 out of the 139 patients in the intervention group for their ability to use the phone: 72 (69.9%) patients were deemed in need of training, and 62 patients subsequently received the training. In total, 29 (46.8%) trainees were capable of using the 3 phone functions (turning the phone on/off, charging the phone, and reading/returning text messages) after the training (more details elsewhere [35]). Information on phone ownership and maintenance, the frequency of phone number changes, and users' experiences and satisfaction are summarized in Table 2. In total, 58 (41.7%) patients and one-fifith of lay health supporters did not have a phone and received a free phone with US$15 prepaid data. In total, 62 out of 63 patients and 77 out of 77 lay health supporters expressed satisfaction with the program, although we cannot conclude that the participants were overall satisfied due to a large amount of missing data (Table 2). Following the same protocol, 8 master's and doctoral students in public health produced a total of 237 educational text messages that covered self-care, medications, symptoms, relapse prevention, rehabilitation, and social resources; 2 messages on relapse signs and medication side effects; and about 150 unique messages of reminders (S2 Appendix). Overall, 27.0% of the families (lay

Tablel. Baseline characteristics.

Characteristic

Count (%) or mean (SD)

Intervention (n = 139)

Control (n = 139)

Patients

Female

77 (55.4%)

77 (55.4%)

Married

87 (62.6%)

90 (64.8%)

Employed

44(31.7%)

48 (34.5%)

Living alone

7 (5.0%)

6 (4.3%)

Age (years)

46.5(12.65)

45.5 (12.72)

Education (years)

7.4 (3.28)

7.1 (3.22)

Literatea

121 (87.1%)

126 (90.6%)

Patient income last month (RMB)b

66(0-500)

95 (0-800)

Family annual income (RMB)b

20,000 (10,000-50,000)

20,000(10,000-50,000)

Duration of schizophrenia (years)

17.5(10.36)

18.4 (10.82)

Caregivers/lay health supporters6

Female

67 (48.2%)

67 (48.2%)

Age (years)

45.4(12.75)

44.5 (12.49)

Employed

76 (54.7%)

62 (44.6%)

Family member of the patient

112 (80.6%)

109 (78.4%)

Married

77 (55.4%)

76 (54.7%)

Patients' health profile

Medication adherenced

Refill recorde

0.75(0.30)

0.71 (0.34)

Drug Attitude Inventory-10 (DAI-10)f

0.65 (0.20)

0.68 (0.20)

Brief Adherence Rating Scale (BARS)g

0.73(0.18)

0.70 (0.20)

Clinical Global Impression (CGI)-Severityh

2.95(1.66)

3.09 (1.70)

WHO Disability Assessment Schedule 2.0 (WHODAS)i

0.18(0.19)

0.19(0.18)

Glasgow Antipsychotic Side-effect Scale (GASS)j

9.47 (6.66)

8.59 (8.02)

Top 5 antipsychotics prescribed

Clozapine

48/136 (35.3%)

45/133 (33.8%)

Risperidone

46/136 (33.8%)

43/133 (32.3%)

Quetiapine

26/136(19.1%)

25/133 (18.8%)

Sulpiride

21/136(15.4%)

25/133 (18.8%)

Perphenazine

12/136 (8.8%)

15/133(11.3%)

aLiterate: defined as no less than 3 years of primary school education.

bIndicated as median (IQR). RMB, renminbi.

cFor the intervention group, these caregivers were recruited as “lay health supporters.”

dPer our research protocol, medication adherence measured by 2 unannounced home pill counts 30 days apart at endpoint was used for the analysis of program effect; however, pill counts were not performed at baseline. Instead, refill records and 2 rating scales were used at baseline.

eAdherence by refill record was calculated as a cumulative medication possession ratio (0%-100%) over 1 year, i.e., number of days medication obtained over 365 days divided by 365 days.

fDAI-10 adherence was originally from -10 to +10 (higher scoreequals more positive attitude toward medication), which was rescaled to be 0 to 1.

gBARS adherence is self-reported percentage of dosages taken over the past month. hHigher scores of CGI-Severity indicate worse symptoms (possible range 1-7).

WHODAS scores indicate the proportion of functioning lost.

jGASS scores indicate patient-reported side effects of antipsychotics: 0-21, no/mild side effects; 22-42, medium side effects; 43 and above, serious side effects.

https://doi.org/1Q.1371/journal.pmed.1QQ2785.tQQ1

Table 2. User experiences in the intervention group.

Experience

Patients (n = 139)a

Lay health supporters (n = 139)b

Phone status

Used a smartphone

33/114 (29.0%)

35/105(33.3%)

Free phone given by LEAN

58/139 (41.7%)

19/139(13.7%)

Changed phone numbers over past 2 months

13/105(12.4%)

92/100 (92.0%)

Phones fully functioning at endpoint

77/99 (77.8%)

92/100 (92.0%)

User evaluation at endpoint

Overall satisfied with the program

62/63 (98.4%)

77/77 (100.0%)

Willing to continue receiving messages

52/57 (91.2%)

80/85 (94.1%)

Messages very useful

61/103 (59.1%)

47/78 (60.3%)

Messages bothered you

4/63 (6.3%)

9/84 (10.7%)

Time of texting appropriate

57/62 (91.9%)

70/77 (90.9%)

Frequency of texting appropriate

53/61 (86. 9%)

66/79 (83.5%)

Length of messages appropriate

59/60 (98.3%)

71/77 (92.2%)

Most useful part of the messages

Treatment and medication education

10/59(17.0%)

18/73 (24.7%)

Family care in schizophrenia

5/86 (8.5%)

8/73 (11.0%)

Medication reminders

27/59 (45.8%)

39/73 (53.4%)

Local news

2/59 (3.4%)

1/71 (1.4%)

Weather forecast

15/59 (25.4%)

7/73 (9.6%)

User capability assessed at endpoint

Able to navigate phone to read messages

52/73 (71.2%)

74/88 (84.1%)

Able to reply to messages

38/73 (52.1%)

55/86 (64.0%)

Did not understand messages

12/68(17.7%)

44/90 (4.9%)

Some physical disability that prevents using a phone

12/65(18.5%)

9/84 (10.7%)

User experiences assessed at endpoint

Always received messages last month

44/71 (62.0%)

65/84 (77.4%)

Always or often read messages

39/71 (54.9%)

65/85 (76.5%)

Frequently replied to texted reminders

15/67 (22.4%)

27/85 (31.8%)

Were concerned about the cost of messages

7/64 (10.9%)

4/83 (4.8%)

Patients: the patients in the intervention group of the program.

bLay health supporters: the lay health supporters for the participants in the intervention group.

https://doi.orq/10.1371/iournal.pmed.1002785.t002

health supporters and/or patients) responded to the medication reminders by texting back “1” every day; 47.0% responded at least once per week. Over the 6 months of follow-up, LEAN cost a total of RMB 53,500 (US$7,926) for the 139 patient participants and 139 lay health supporters in the intervention group, which included RMB 19,000 (US$2,815) for texting fees, RMB 7,600 (US$1,126) for the message development, RMB 4,800 (US$711) for the message management, RMB 10,000 (US$1,481) for the 77 phones provided to the patients and the lay health supporters, and RMB 9,000 (US$1,333) for the additional time cost for the health workers.

Primary outcome: Adherence

We note strong evidence of an intervention effect on adherence to antipsychotic medications. Medication adherence measured by the unannounced home-based pill counts was 27% greater in the intervention group (0.61) than in the control group (0.48) (adjusted mean difference [AMD] 0.12 [95% CI 0.03 to 0.22]; p = 0.013; effect size 0.38; Table 3; Fig 2). Our study was


Table 3. Primary and secondary outcomes at 6 months.

Measure


Mean difference (95% CI) or relative risk (95% CI)


p-Value


Mean (SD) or n/N (%)

Intervention (n = 139)

Control (n = 139)

Primary outcome

Pill-count adherencea

0.61 (0.34)

0.48 (0.35)

0.12 (0.03 to 0.22)b

0.013

Other adherence measurements

DAI-10c

0.68 (0.20)

0.67 (0.22)

0.02 (-0.05 to 0.08)

0.67

BARSd

0.71 (0.21)

0.68 (0.23)

0.03 (-0.04 to 0.10)

0.37

Refill recorde

0.83 (0.28)

0.76 (0.34)

0.04 (-0.01 to 0.10)

0.12


Secondary outcomes

WHODASf

0.12(0.15)

0.15(0.19)

-0.03 (-0.07 to 0.01)b

0.117

CGI-Severityg

2.84 (1.37)

2.76(1.24)

0.11 (-0.21 to 0.42)b

0.514

Negative

2.94(1.46)

2.98 (1.43)

0.02 (-0.29 to 0.32)

0.908

Positive

2.70(1.62)

2.67 (1.55)

0.17 (-0.14 to 0.49)

0.277

Depression

2.31 (1.29)

2.11 (1.26)

0.75 (-0.15 to 0.30)

0.517

Cognition

2.86(1.50)

2.85(1.44)

0.07 (-0.22 to 0.36)

0.617

CGI-Improvementh

3.09(1.15)

3.02 (1.08)

0.03 (-0.25 to 0.30)

0.848

Other outcomes from the “686 Program”1

Relapsej

26/120 (21.7%)

40/117(34.2%)

0.63 (0.42 to 0.97)

0.033

Re-hospitalization due to schizophrenia

9/123 (7.3%)

25/122 (20.5%)

0.36 (0.17 to 0.73)

0.004

Death for any reason

2/139(1.4%)

1/134 (0.8%)

1.93 (0.18 to 21.01)

0.590

Substance abuse

14/133 (10.5%)

13/127(10.2%)

1.028 (0.50 to 2.10)

0.939

Suicide

0/139 (0%)

0/139 (0%)

Self-harm

0/139 (0%)

0/139 (0%)

Wandering

2/138(1.5%)

2/134(1.5%)

0.97 (0.14 to 6.79)

0.976

Violence against others

1/137(0.7%)

2/134(1.5%)

0.49 (0.04 to 5.33)

0.557

Damaging goods

2/138(1.5%)

5/134 (3.7%)

0.39 (0.08 to 1.97)

0.252

aProportion of antipsychotic dosages taken over the past month assessed by unannounced home-based pill counts (possible range 0-1). bAdjusted mean difference.

cDrug Attitude Inventory-10 (DAI-10) adherence was originally from -10 to +10 (higher score = more positive attitude toward medication), which was rescaled to be 0 to 1.

dBrief Adherence Rating Scale (BARS) is self-reported proportion of dosages taken over the past month.

eRefill record adherence is number of days medication obtained over past 182 days divided by 182 days.

fWHO Disability Assessment Schedule 2.0 (WHODAS): proportion of functioning lost (possible range 0-1).

gClinical Global Impression (CGI)-Severity: Higher scores indicate worse symptoms (possible range 1-7).

hCGI-Improvement indicates degree of change in symptoms (1 = very much improved; 2 = much improved; 3 = minimally improved; 4 = no change; 5 = minimally worse; 6 = much worse; 7 = very much worse).

These outcomes were tracked by the 686 Program administrative system on a routine basis. There was a small number of missing data.

jRelapse is defined as an overall and marked increase in symptoms as reassessed by health professionals through interviewing patients and family members.

https://doi.org/10.1371/journal.pmed.1002785.t003

underpowered to detect treatment interaction with baseline adherence (p for interaction 0.99), although we found a similar intervention effect on adherence within the subset who were nonadherent at baseline (mean adherence at endpoint 0.59 in the intervention group versus 0.46 in the control group; AMD 0.13 [95% CI 0.00 to 0.25];p = 0.047; effect size 0.36; Table 3; Fig 2), while the program effect attenuated for the baseline adherent group (AMD 0.08 [95% CI -0.06 to 0.22];p = 0.265). In total, 59 out of278 participants (21.22%) had missing outcomes (S1 Appendix). Meanwhile, the participants in LEAN attended a mean of83% of scheduled clinical appointments, higher than the 76% in the control group (p = 0.066).


Adherence-endpointa

Overall group

Adherence(baseline)d adherent nonadherent

WHODAS (baseline)

>0.22

<0.22

WHODAS-endpointb

Overall group

Adherence(baseline)d adherent nonadherent WHODAS (baseline)

>0.22

<0.22

CGI-endpointc

Overall group

Adherence(baseline)d adherent nonadherent WHODAS (baseline)

>0.22

<0.22


Intervention

Control

Mean

SD

Mean

SD

0.61

0.34

0.48

0.35

0.63

0.31

0.51

0.34

0.59

0.37

0.46

0.36

0.58

0.39

0.41

0.37

0.63

0.32

0.53

0.34

0.12

0.15

0.15

0.19

0.09

0.13

0.19

0.22

0.14

0.17

0.11

0.17

0.23

0.21

0.20

0.21

0.06

0.09

0.10

0.15

2.84

1.37

2.76

1.24

2.62

1.15

2.92

1.30

3.03

1.51

2.63

1.19

3.43

1.50

3.27

1.42

2.64

1.30

2.63

1.10


Adjusted mean difference Adjusted mean difference P value (95%CI)             ‘


(95%CI)


0.12

[0.03;

0.22]

0.013

0.08

[-0.06; 0.22]

0.265

0.13

[0.00;

0.25]

0.047

0.20

[0.00;

0.39]

0.050

0.05

[-0.08;

0.17]

0.460

-0.03

[-0.07;

0.01]

0.117

-0.08

[-0.15;-

-0.01]

0.017

0.01

[-0.04;

0.07]

0.603

0.03

[-0.09;

0.15]

0.608

-0.03

[-0.07;

0.01]

0.154

0.10

[-0.21;

0.42]

0.514

-0.17

[-0.63;

0.30]

0.485

0.33

[-0.10;

0.76]

0.135

0.14

[-0.54;

0.83]

0.683

0.02

[-0.35;

0.40]

0.903


P value for interaction


0.99


0.43


0.08


0.96


0.12


0.22


-0.5

0.5


Fig 2. Subgroup analysis. Adherence endpoint is proportion of antipsychotic dosages taken over a month assessed by 2 unannounced home-based pill counts 30 days apart at 6 months (possible range 0-1). bWHODAS endpoint is WHO Disability Assessment Schedule 2.0, indicating proportion of loss of functioning at 6 months (possible range 0-1). cCGI endpoint is Clinical Global Impression-Severity assessed at 6 months (possible range 1-7). dAdherence (baseline) is based on medication refill adherence over the past year at baseline: people who collected all 6 refills were considered adherent, and those who missed any of the 6 refills were considered nonadherent.

https://doi.orq/10.1371/iournal.pmed.1002785.q002

Secondary outcomes: Functioning and symptoms

There was slightly less loss of functioning in the intervention group than in the control group, though the difference was not statistically significant (mean WHODAS score 0.12 in the intervention group and 0.15 in the control group; AMD -0.03 [95% CI -0.07 to 0.01];p = 0.117; effect size 0.18; Fig 2). There is, however, evidence of effect modification with baseline medication adherence (p for interaction 0.08): For the subset with good medication adherence at baseline, the mean WHODAS score was 0.19 in the intervention group and 0.09 in the control group (AMD -0.08 [95% CI -0.15 to -0.01];p = 0.017; effect size 0.57; Fig 2); however, there was no significant difference in patient functioning between the groups for the subset with poor baseline adherence (AMD 0.01 [95% CI -0.04 to 0.07]; p = 0.603; Fig 2). We did not note any significant improvement in the severity of symptoms for the overall group or the pre-specified subgroups (Fig 2).

Other outcomes

There was strong evidence of a substantial reduction in the risk of relapse (26 [21.7%] of 120 interventional participants versus 40 [34.2%] of 117 controls; relative risk [RR] 0.63 [95% CI 0.42 to 0.97]); number needed to treat [NNT] 8.0 [95% CI 4.2-85.2]) and the risk of re-hospitalization (9 [7.3%] of 123 interventional participants versus 25 [20.5%] of 122 controls; RR 0.36 [95% CI 0.17 to 0.73]; NNT 7.6 [95% CI 4.6-21.3]) with the intervention (Table 3).

Raw versus adjusted analyses

We performed a sensitivity analysis to compare the results of the program effects on adherence, functioning, and symptoms with a raw analysis versus an adjusted analysis with covariates and data imputation for the missing data. The results were not sensitive to the choice of the methods, with almost identical results for adherence and functioning and a minor difference for symptoms (see S4 Appendix).

Discussion

In this study, we used a 2-arm randomized controlled trial to study the effect of mobile texting on medication adherence, functioning, and symptoms of community-dwelling people with schizophrenia in rural China. Our trial showed that the addition of texting to patients and their lay health supporters in a resource-poor community setting compared with a free-medi-cine program alone improved medication adherence (0.48 in the control group versus 0.61 in the intervention group; effect size 0.38) and substantially reduced relapses and re-hospitalizations, but our program did not lead to significant changes in patient functioning or symptoms. The program was also found to be generally well accepted by the patients and their families, was relatively easy to implement and use, and added little marginal cost.

The existing evidence of the effect of texting on adherence, functioning, and symptoms is conflicting [15]. Six randomized controlled trials were identified that used texting for people with schizophrenia [15,50]: A recent trial in Finland (n = 1,139) showed no advantages of texting on any outcomes assessed at 12 months [51], which conformed to the results of 2 earlier trials (Netherlands [n = 62] [52]; Czech Republic [n = 146] [53]). One trial carried out in Spain (n = 254) [54] and 2 US trials (n = 30 and n = 55) [55,56], however, found significant improvement in medication adherence and some reduction in symptoms. Few studies reported adequately on outcomes related to patient functioning.

We would like to discuss 4 aspects of LEAN relative to prior studies. First, LEAN demonstrated a 27% relative improvement in adherence, which is larger than the 15%-18% range reported in other text message interventions [27]. Meanwhile, our program improved patients' attendance at scheduled clinical appointments. Prior studies found mixed effects of the use of technological prompts on appointment attendance in psychological settings [57,58]. Three unique features may have contributed to the relative superiority of LEAN: (1) active engagement of the lay health supporters, (2) the varying contents of our medication reminder, which probably reduced receivers' fatigue compared to other studies [59], and (3) the use of texting to connect and integrate the entire treatment team, from patients to the lay health supporters to the village doctors to the psychiatrists, all in support of the patient. In line with the theory of the HBM [33], text reminders and lay health supporters may have provided “cues to action” to address forgetfulness and reluctance to take medicine [30], while the texted education may have improved the perceived net benefits of the medications. There was improved attitude toward medication as shown by DAI-10 score, although the action cues probably played a bigger role—45.8% of patients regarded text medication reminders as most useful, while only 17.0% considered educational messages most useful.

Second, like most of the 6 randomized controlled trials discussed earlier [15,50], the improvement in medication adherence did not lead to significant reported changes in symptoms. Perhaps there was a ceiling effect, as the program participants in those studies and ours on average had only mild symptoms at baseline (Table 1). It may also be possible that the low adherence, even after LEAN, prevented the medicine from releasing its full potency. Even so, the substantial reduction in relapses (RR 0.63) and re-hospitalizations (RR 0.36) may indicate

that, despite lack of effect for the whole group, there may be some program effect on symptoms for certain subsets of patients.

Third, prior studies of texting for schizophrenia seldom reported the global functioning level of the patients. Our program had a small and statistically insignificant effect on reported patient functioning for the overall group (effect size 0.17; p = 0.117), but it had a medium and significant effect for the subset with good baseline adherence (effect size 0.57; p = 0.017). As this improvement in the subset was not accompanied by improvement in medication adherence, we suspect that the text messages may have served as a rudimentary psychosocial intervention that was beneficial to functioning. Earlier studies indicated even simple messages asking “how are you?” or saying “thank you” reduced social isolation and improved functioning [60]. However, we should note that this subgroup effect may simply result from pure chance.

Finally, our program appears to have achieved a good level of participant satisfaction, and our program attrition was only 4.3%, compared with an overall rate of 20.0% (95% CI 17% to 24%) [61] in interventional trials for schizophrenia.

The trial used a waitlist control design whereby the control group would receive the intervention as well once the program demonstrated benefits after the initial 6-month implementation. We suspended LEAN from August 2017 to March 2018 due to our program evaluation. We resumed LEAN in both the original intervention and the waitlist control groups from April to October 2018. We are now in the process of cleaning the data from this extended program phase and will report the results in subsequent publications (program updates are available from https://www.researchgate.net/project/LEAN-Trial-Lay-Workers-mHealth-for-Severe-Mental-Disorders).

A special issue concerning the use of clozapine is worth discussion. Over 30% of our program participants used clozapine. Because of the close monitoring needed with this medication, the use of clozapine itself may increase treatment adherence and reduce symptoms. However, as the use of clozapine between our intervention and control groups was balanced at baseline (35% versus 33%) (Table 1), the use of clozapine should not lead to bias in our assessment of the program effects. Further, LEAN might help improve the use of clozapine because of the enhanced education on side effects and the facilitated communication between the patient families and the health professionals through texting, for quicker medication adjustment in between routine psychiatrists' visits every 2 months.

Many lessons learned from this trial can be potentially useful for other LMICs that face resource constraints. China's 686 Program successfully implemented many WHO mhGAP recommendations for resource-poor settings. In particular, the 686 Program in Liuyang effectively removed the access barriers for antipsychotics by providing free medication routinely and conveniently. However, adherence to antipsychotics remained a serious challenge. Our texting intervention further improved the program by addressing the low adherence at marginal cost. Elements of LEAN may be adapted to other resource-poor settings with or without an existing community-based program. However, adaptation of LEAN should fully consider some implementation details including (1) keeping the program simple and integrated into routine care [62,63] (LEAN required minimal training and leveraged existing resources such as a commercial telemarketing platform and the 686 Program structures), (2) maintaining low cost (LEAN cost a total of US$7,926 for the 139 patient participants and 139 lay health supporters), (3) having a reliable system to track changes of phone numbers (participants frequently changed numbers [Table 3]), and (4) choosing the right phones (some cheap phones' small storage filled up quickly and prevented incoming messages). Furthermore, long-acting injectable antipsychotics were not available through the 686 Program. Both the clinicians and families perceived the injectable to be unpredictable and less safe. The 686 Program should

develop a guideline on the use of those long-acting injectables, particularly among people with low adherence to pills. We should also note that adherence measured by clinician impression and refill records (the current 686 Program practice) grossly overestimated the level of adherence; the use of simple scales such as DAI-10 or BARS may be considered if home-based pill counts are not feasible [44]. Lastly, we emphasize proper training for the participants on how to receive, read, and reply to text messages. Despite our training, at the endpoint, 28.8% of patients still could not read the messages. To address this challenge, we sent voice messages for some participants and simultaneously texted the lay health supporters. Our experiences caution against the use of more complicated smartphones among people with schizophrenia with low education in LMICs.

There are several limitations to our trial. First, certain features of the 686 Program may limit the generalizability of the findings to other parts of the world, given that other populations may have limited access to medications, or distinct cultural beliefs about the origins and meaning of mental illness. However, the study should provide solid reference points for programs considering the use of texting and lay health supporters. Second, we investigated our program's cost, but we did not perform a cost-effectiveness analysis. This partially reflected the preference of the local policy-makers for low cost rather than cost-effectiveness. Third, our pursuit of simplicity sacrificed the ability to customize the content, frequency, and timing of the messages to individual patients. Individual tailoring is considered more effective [15,17] but would have significantly increased program complexity and cost. Fourth, our trial only had a 6-month follow-up, and thus we could not determine the longer-term effects of the intervention on adherence, symptoms, and functioning. Fifth, despite our best efforts to capture adherence, the unannounced pill count can still be subject to inaccuracy. In particular, the number of discarded and additionally obtained pills as reported by the patients and their family members may be inaccurate because of memory lapses or by intention. However, this possible inaccuracy may not lead to bias as its effects may be canceled out between the intervention and control groups due to randomization. Sixth, due to a program administrative mishap, we failed to assess medication side effects at endpoint as our protocol had specified. We thus had no information on the effect of LEAN on medication side effects. Seventh, we could assess the overall effects of LEAN on adherence, functioning, and symptoms, but the effects could not be attributed to specific program elements (e.g., how much of the program's effects can be attributed to our text medication reminders to the lay health supporters or the patients?) Eighth, there was a possible risk of bias if family members were present during our patient assessment in a different fashion between the 2 groups. We tried hard to stick to the same assessment protocol to reduce this risk. Finally, our program possibly missed some of the least adherent people as we excluded those missing the 3 past medication refills. This de facto withdrawal from the 686 Program may be for reasons such as feeling highly functioning and deciding to discontinue medications, choosing to purchase medications outside of the 686 Program, or intentionally or unintentionally missing refills due to sickness.

Our study points to several future directions for research. Some non-schizophrenia studies have suggested that less frequent messages are more effective [15]. Future trials should test that possibility. Furthermore, 33.3% of lay health supporters and 29.0% of patients used a smartphone. Smartphones, with their sensor technologies and apps, may have considerable potential for improving the health of people with schizophrenia [64]. However, complicated apps may create barriers as well, considering that 28.8% and 47.9% of our patients could not even master the simple task of reading and replying to messages, respectively. The role of smartphones needs to be further explored in trials. Finally, potential adverse effects of text messaging on patients and their families should be more thoroughly investigated. Four (6.3%) patients and 10 (10.7%) lay health supporters did report texting bothered them.

Supporting information

S1 CONSORT. CONSORT checklist.

(DOC)

(DOCX)

(DOCX)

Acknowledgments

We thank the patient and family participants; Liuyang Mental Health Center (Meng Dai, Li Chen) for providing patient assessment and policy support; mental health administrators from 9 THCs (Zhong Huang, Huakun Li, Yanjiang Li, Xianyong Li, Jiaona Liu, Change Lu, Hao Luo, Dong Pan, Xiao Pan, Huiyun Shao, Shuyi Tang, Xiang Wang, Ya Xu, Huiming Zhang, and Qing Zeng) for coordinating and assisting in our field work and home visits; and the LEAN project volunteer group for the development and management of text messages, field data collection, and management.

The project volunteers were Sanmei Chen, Fei Deng, Chunli Fang, Weijie Gong, Gang Hu, Fengsu Hou, Jiajun Jing, Di Liang, Quanlei Li, Hengzhuo Liu, Di Liu, Tianlin Ma, Yushi Mo, Bingwei Tang, Nan Zhang, Hui Wang, Xueyuan Wang, Yunfang Wang, Anwen Yang, Chao Zhang, Donglan Zhang, Mei Zhao, Qiufen Zhu, and Tianjiao Zhu.

Author Contributions

Conceptualization: Dong (Roman) Xu, Shuiyuan Xiao, Eric D. Caine, Stephen Gloyd, Jane Simoni, James P. Hughes, Wenjie Gong.

Data curation: Wenjun He.

Formal analysis: Dong (Roman) Xu, Hua He, Stephen Gloyd, Jane Simoni, James P. Hughes, Wenjun He, Wenjie Gong.

Funding acquisition: Dong (Roman) Xu, Wenjie Gong.

Investigation: Dong (Roman) Xu, Juan Nie, Meijuan Lin, Yeqing Yuan, Wenjie Gong.

Methodology: Dong (Roman) Xu, Hua He, James P. Hughes, Wenjie Gong.

Project administration: Dong (Roman) Xu, Juan Nie, Meijuan Lin, Yeqing Yuan, Wenjie Gong.

Resources: Dong (Roman) Xu, Shuiyuan Xiao, Eric D. Caine, Stephen Gloyd, Wenjie Gong.

Software: Wenjun He.

Supervision: Dong (Roman) Xu, Wenjie Gong.

Validation: Hua He.

Visualization: Wenjun He.

Writing - original draft: Dong (Roman) Xu, Wenjie Gong.

Writing - review & editing: Dong (Roman) Xu, Shuiyuan Xiao, Hua He, Eric D. Caine, Stephen Gloyd, Jane Simoni, James P. Hughes, Juan Nie, Meijuan Lin, Wenjun He, Yeqing Yuan, Wenjie Gong.

References

PLOS Medicine | https://doi.org/10.1371/journal.pmed.1002785 April 23, 2019

21 /21

1

Corresponding author at: Centre for Youth Mental Health, The University of Melbourne, Orygen Youth Health Research Centre, 35, Poplar Road, Parkville 3054, Victoria, Melbourne Australia. Tel.: +61 9342 2805, +61 401772668 (mobile).

E-mail address: malvarez@unimelb.edu.au (M. Alvarez-Jimenez).

2

Corresponding author. 35 Poplar Rd, Parkville VIC, 3052, Australia. E-mail addresses: imogen.bell@orygen.org.au (I.H. Bell), neilthomas@swin.edu. au (N. Thomas).

https://doi.org/10.1016/j.schres.2019.10.026

0920-9964/© 2019 Elsevier B.V. All rights reserved.

3

Background: Numerous psychosocial interventions for individuals with chronic psychotic disorders (CPD) have shown positive effects on social cognitive and functional outcome measures. However, access to and engagement with these interventions remains limited. This is partly because these interventions require specially trained therapists, are not available in all clinical settings, and have a high scheduling burden for participants, usually requiring a commitment of several weeks. Delivering interventions remotely via mobile devices may facilitate access, improve scheduling flexibility, and decrease participant burden, thus improving adherence to intervention requirements. To address these needs, we designed the Creating Live Interactions to Mitigate Barriers (CLIMB) digital intervention, which aims to enhance social functioning in people with CPD. CLIMB consists of two treatment components: a computerized social cognition training (SCT) program and optimized remote group therapy (ORGT). ORGT is an innovative treatment that combines remote group therapy with group texting (short message service, SMS).

4

Objectives: The objectives of this single-arm study were to investigate the feasibility of delivering 6 weeks of CLIMB to people with CPD and explore the initial effects on outcomes.

5

Methods: Participants were recruited, screened and enrolled via the Internet, and delivered assessments and interventions remotely using provided tablets (iPads). Participants were asked to complete 18 hours of SCT and to attend 6 remote group therapy

6

sessions. To assess feasibility, adherence to study procedures, attrition rates, engagement metrics, and acceptability of the intervention were evaluated. Changes on measures of social cognition, quality of life, and symptoms were also explored.

7

Results: In total, 27 participants were enrolled over 12 months. Remote assessments were completed successfully on 96% (26/27) of the enrolled participants. Retention in the 6-week trial was 78% (21/27). Of all the iPads used, 95% (22/23) were returned undamaged at the end of the intervention. Participants on average attended 84% of the group therapy sessions, completed a median of 9.5 hours of SCT, and posted a median of 5.2 messages per week on the group text chat. Participants rated CLIMB in the medium range in usability, acceptability, enjoyment, and perceived benefit. Participants showed significant improvements in emotion identification abilities for prosodic happiness (P=.001), prosodic happiness intensity (P=.04), and facial anger (P=.04), with large within-group effect sizes (d=.60 to d=.86). Trend-level improvements were observed on aspects of quality of life (P values less than .09). No improvements were observed for symptoms.

8

* Address for correspondence: Prof. G. Donohoe, School of Psychology and Center for Neuroimaging and Cognitive Genetics, National University of Ireland, Galway, Ireland.

(Email: donoghug@tcd.ie)

9

These authors contributed equally to this work.

10

* Corresponding author.

11

These authors contributed equally to this work.

12

Correspondence to: VA Pittsburgh Health Care System, University Drive C, Behavioral Health (116a), Pittsburgh, PA 15240, United States.

E-mail address: kasckowjw@upmc.edu (J. Kasckow).

13

Corresponding author.

E-mail address: m.krzystanek@sum.edu.pl (M. Krzystanek).

14

There is no known threshold for problematic or severe loneliness but a score of 38 and above was used to indicate above the median score across different samples (Russell [58]).

15

A rating of moderate or more on any positive psychotic symptoms as assessed by the Positive and Negative Syndrome Scale (Kay et al. [52]).

16

Assessed by the SCID 5 risk section.

17

Manuscript submitted for publication. Contact first author of manuscript and group program.

18

Points were provided to participants when they correctly answered a question, but no penalty was given for incorrect responses. Challenges involved participants relating the information and skills learnt during the daily tasks to real-world situations. An example of a challenge was responding in an active and constructive manner to someone when they heard positive news. Badges were assigned for either progression through the app, completing the mood log over a particular period, or completion of a challenge.

19

Measures were administered at all timepoints except for the SCID-5-RV and NART which were administered only once at T1.

20

21

The interview schedule is available upon request from the first author.

22

  Iverson Health Innovation Research Institute, Swinburne University of Technology, Hawthorn, Melbourne 3122, Australia

23

  Centre for Mental Health, Swinburne University

of Technology, Hawthorn, Melbourne 3122, Australia

24

  School of Behavioural and Health Sciences, Australian Catholic University, Sydney, Australia

25

| INTRODUCTION

Recovery from mental illness is an evolving concept in the field of psychology, especially for individuals early in the course of psychosis (Roe, Mashiach-Eizenberg, & Lysaker, 2011). While common objective measures of recovery from psychosis include functional outcomes (eg, employment status) and symptom remission, subjective indicators of

26

An affirmative response indicates a non-adherence attitude. MAQMorisky Green Adherence Questionnaire.

27

Previous presentation: The article has not been published before nor is under consideration elsewhere.

* Corresponding author.

E-mail address: moritz@uke.uni-hamburg.de (S. Moritz).

28

Split first authors.

29

Split senior authors.

30

Psychiatry and Mental Health Department, Centro Hospitalar Universitario Lisboa Norte; fFaculty of Medicine, University of Lisbon, Lisboa, Portugal; ^Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; §Psychiatry Department, Centro Hospitalar Lisboa Ocidental, Lisboa; and || Department of Psychiatry, Hospital Garcia de Orta, Almada, Portugal.

Bernardo Melo Moura and Alessia Avila contributed equally to this work.

Send reprint requests to Bernardo Melo Moura, MD, Servigo de Psiquiatria e Saude Mental, Hospital de Santa Maria, Av. Professor Egas Moniz, 1649-035 Lisboa, Portugal. E-mail: bernardomoura@campus.ul.pt

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.jonmd.com).

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.

ISSN: 0022-3018/19/20711-0951

DOI: 10.1097/NMD.0000000000001055

31

Abbreviations: SZ, Schizophrenia; HC, Healthy Control.

32

Corresponding author. Mor Nahum, Posit Science, 77 Geary St., Suite 303, San Francisco, CA 94108. Tel.: +1 415 269 2425.

E-mail address: mor.nahum@positscience.com (M. Nahum).

2215-0013/$ - see front matter © 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.scog.2014.01.003

33

Correspondence should be addressed to Dr Marieke Pijnenborg, Department of Psychotic Disorders, GGZ Drenthe, Dennenweg 9, 9404 LA Assen, The Netherlands (e-mail: marieke.pijnenborg@ggzdrenthe.nl).

DOI:10.1348/014466509X467828

34

Correspondence: goodmansibeKo@gmail.com

35

Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa

Full list of author information is available at the end of the article

36

Corresponding author at: Servicio de Psiquiatria, Complexo Hospitalario Universitario de Ourense (CHUO), Hospital Santa Maria Nai, Rua Ramon Puga 52-55, 32005, Ourense, Spain.

E-mail address: alejandro.alberto.garcia.caballero@sergas.es (A. Garcia-Caballero).

37

The reliability (r) and validity (v) values for the AIHQ, Faux Pas and Hinting Task scales are expressed as mean and standard deviation (on a scale from 1 to 9, according to expert consensus), and the other measures are expressed according to the corresponding coefficient. The Happe test, despite being one of the classical tasks in assessing ToM, has no data on r or v.